Orbiter History

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [X....X..............X....X..................X...............] duration: 5.421500676s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:43:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.421522087s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..............................X...X.........................] duration: 3.947847756s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:43:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.947868615s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.X.......................] duration: 6.868099807s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:43:11 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.868113452s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [.X....X..............X....X..................X..............] duration: 4.105448004s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:43:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.105458703s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.X......................] duration: 6.693330381s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:42:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.693347384s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...............................X...X........................] duration: 4.936693235s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:42:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 4.936712603s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [..X....X..............X....X..................X.............] duration: 4.265910005s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:42:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.265934721s core uplink uk-select-1 uk1-placement-17
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [...X....X..............X....X..................X............] duration: 4.160017634s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:41:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.160032853s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.X.....................] duration: 6.874709818s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:41:31 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.87472662s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [....X....X..............X....X..................X...........] duration: 3.960239162s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:41:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.960264299s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [................................X...X.......................] duration: 3.818675608s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:41:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.81869271s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [.....X....X..............X....X..................X..........] duration: 3.945010183s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:40:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.945030253s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.X....................] duration: 6.727348154s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:40:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.727368564s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [......X....X..............X....X..................X.........] duration: 3.971853958s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:40:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.971882303s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.................................X...X......................] duration: 3.914630811s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:40:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.914725741s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [.......X....X..............X....X..................X........] duration: 3.985623181s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:40:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.98567564s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..................................X...X.....................] duration: 3.779948414s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:40:03 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.779972601s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.X...................] duration: 6.629228783s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:39:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.629242348s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [........X....X..............X....X..................X.......] duration: 4.028512375s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:38:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.028534015s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.X..................] duration: 6.607446641s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:38:57 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.607491025s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...................................X...X....................] duration: 3.883037619s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:38:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.883047718s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [.........X....X..............X....X..................X......] duration: 4.041033053s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:38:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 4.041070735s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [....................................X...X...................] duration: 3.785609075s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:37:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.785621019s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.X.................] duration: 6.87970395s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:37:44 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.879722947s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [..........X....X..............X....X..................X.....] duration: 5.657411369s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:37:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.657435224s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.....................................X...X..................] duration: 3.859899966s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:37:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.859979318s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [...........X....X..............X....X..................X....] duration: 3.979959827s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:36:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.979983873s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.X................] duration: 6.666537107s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:36:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.666556986s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [............X....X..............X....X..................X...] duration: 3.964112576s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:36:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.9641304s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [......................................X...X.................] duration: 3.831405956s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:36:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.831426776s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.X...............] duration: 6.803235913s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:36:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.803259018s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [.............X....X..............X....X..................X..] duration: 3.965116722s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:36:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.965138313s core uplink uk-select-1 uk1-placement-17
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 6 out of 60 is failed (threshold=3) [X.............X....X..............X....X..................X.] duration: 3.979460172s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:35:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 3.97948046s core uplink uk-select-1 uk1-placement-17
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 6 out of 60 is failed (threshold=3) [.X.............X....X..............X....X..................X] duration: 1m0.976379106s trace: https://jaeger.dev.storj.io/trace/591f717acd2fb4ee FAIL (2026-05-26 11:35:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== ]: Uploading piece... May 26 11:35:11.676332990 (33h30m57.459673089s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.676655064 (33h30m57.459995163s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677013428 (33h30m57.460353517s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677370369 (33h30m57.460710468s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.677728092 (33h30m57.461068191s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678714394 (33h30m57.462054493s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.678871233 (33h30m57.462211332s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.679685467 (33h30m57.463025566s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.680638816 (33h30m57.463978905s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.682491591 (33h30m57.46583169s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.683877145 (33h30m57.467217244s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.685180713 (33h30m57.468520812s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.686130345 (33h30m57.469470444s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687102040 (33h30m57.470442129s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:11.687332258 (33h30m57.470672357s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Uploading piece... May 26 11:35:12.741032974 (33h30m58.524373073s) [upload=32098]: Next returned nil segment May 26 11:35:12.741045958 (33h30m58.524386057s) [upload=32098]: Waiting for error group managing segments... May 26 11:35:13.019471619 (33h30m58.802811718s) [noise=true , address=212.102.52.214:4358 , piece_id=MR2A7DMYMP7KBW6AMYAOA7SQAPAZBZP23CBUDMOCM72UEH6DITZQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.021129593 (33h30m58.804469702s) [noise=true , address=212.102.52.213:4326 , piece_id=7QK6W2FY635YCZ65MGRU65HHET7GEMFEKRRMOUMLRLXWDMHM443A, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.031315600 (33h30m58.814655699s) [noise=true , address=212.102.52.215:4378 , piece_id=EDH47ZYURW5MQM6KE6H67NJ5TVADBXPTE53BCM3T6L4DM4F4MBZA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.033023238 (33h30m58.816363337s) [noise=true , address=212.102.52.215:4386 , piece_id=NYRONC24MXS3EXLC6BKVXFD3UCHCI7DODADA3EPHGAHMVYSPBVXA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.037750649 (33h30m58.821090748s) [noise=true , address=212.102.52.213:4332 , piece_id=KMD3HPIIZTDGCQLIXAIZDHEIFPDCTCPPPINHUDISNGH2Z6HLFNIQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.040935426 (33h30m58.824275535s) [noise=true , address=212.102.52.214:4364 , piece_id=5DKUEXVP5MGT6CFRMFUJMIECZDKF3EVT6I4CAQE7ITMCTNRAFIYA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.043623124 (33h30m58.826963224s) [noise=true , address=212.102.52.217:4439 , piece_id=NYK36DFFJ274Y2MONVJYYA4PK6N2AAJF6G4AB6MCK2T65YGJ4GRA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044186480 (33h30m58.827526569s) [noise=true , address=212.102.52.217:4428 , piece_id=PIHZ7VDAAYIEBEZXIHLPLKWBS6DNBBMBB5XVGZFPDWUHFBRBXFJQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.044839977 (33h30m58.828180076s) [noise=true , address=212.102.52.217:4433 , piece_id=DXX345PZTA4NDQI6ITGSPPSR2RMUDOCEBOVEOT77QZ4SUDY7BJ2Q, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.050906402 (33h30m58.834246521s) [noise=true , address=212.102.52.213:4343 , piece_id=X25IK7IPQLB32FX4VIFJABXI2KGGYA52CJZYMX75ZUKM4RDMAFRQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056117936 (33h30m58.839458036s) [noise=true , address=212.102.52.214:4345 , piece_id=ZNRIBWRLWN6OGFXO6O73CPBCI4G5OVNPXSWSKGRC6L64EBVOGQDQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.056778046 (33h30m58.840118155s) [noise=true , address=212.102.52.216:4398 , piece_id=BV3ATTSW2A7VBWKTX4DONDJFOPSKULSMZDIPR23MMRGIKV2XCFKQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.057824563 (33h30m58.841164662s) [noise=true , address=212.102.52.214:4367 , piece_id=EO7DKKYTQY3ERXHUMK3PHMV3SDG7GXJLKL7A4FSWVNYKV5H43JVQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.059053168 (33h30m58.842393257s) [noise=true , address=212.102.52.216:4415 , piece_id=V3CGPGQRDE6LZE7IKO6QYXNEX76UZXPILDTHKSBNOBTGRL3B36EQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.090809665 (33h30m58.874149764s) [noise=true , address=212.102.52.217:4440 , piece_id=URXT4ALA6VS2MBG7DNYICBKJCYONVKDU6NGFQUKNFKUWRMBIMTPA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096832378 (33h30m58.880172478s) [noise=true , address=212.102.52.215:4380 , piece_id=EN7N2NR5TGGWCY7CLSBQJTKYHWPMZLLE5ZN4UAP3AERXPVMSFNKA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: <nil> May 26 11:35:13.096851434 (33h30m58.880191533s) [seg_pos={0 0}, upload=32098]: Satisfied cohort requirements after 16 pieces May 26 11:35:13.097096842 (33h30m58.880436941s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097104908 (33h30m58.880445008s) [noise=true , address=212.102.52.216:4416 , piece_id=BVPXBFWS7MWK2QQISFBBYPBCXLIZKBXSY7B72WQU47CGAK4SU6JQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097299419 (33h30m58.880639538s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097314168 (33h30m58.880654267s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GobdbvLireyEB6WtiokfHU1PnDtmyoGTZctB7PTzcjG7Spz1z): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097324467 (33h30m58.880664566s) [noise=true , address=212.102.52.216:4393 , piece_id=VQRYNNP2HH35JPBO23M4N3LNTYIZ42KAQ3TY4BWPWJKBIUBPXZNQ, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097331600 (33h30m58.8806717s) [noise=true , address=212.102.52.213:4322 , piece_id=GKO5QOE52VYHZ55S4MODTYTP35FHEV3X3GZ6LGAMMYCBAP64LSSA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097455767 (33h30m58.880795856s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KAQ6XCg6V2H4a7xAsWDtXLsfbfVs3DTVFHTWUGUnwQLkF79EK): protocol: expected piece hash; context canceled; EOF May 26 11:35:13.097461388 (33h30m58.880801487s) [noise=true , address=212.102.52.215:4390 , piece_id=QOLA7LL4VRYSJW2M6MKCSMGHNTEZWJAHFKFDRELS3EGVL4K4ELMA, seg_pos={0 0}, upload=32098]: Long tail cancellation due to optimal threshold May 26 11:35:13.097499872 (33h30m58.880839971s) [seg_pos={0 0}, upload=32098]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 26 11:35:13.097564916 (33h30m58.880905015s) [upload=32098]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 26 11:35:13.097569725 (33h30m58.880909824s) [upload=32098]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 26 11:35:37.962376623 (33h31m23.745716722s) [upload=32098]: Done uploading segments... May 26 11:35:37.962419524 (33h31m23.745759623s) [upload=32098]: Upload finished. err: metaclient: context deadline exceeded Duration: 1m0.976395467s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.X..............] duration: 7.356614588s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:35:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.356639235s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.......................................X...X................] duration: 3.848724994s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:35:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.848747618s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.X.............] duration: 6.62907445s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:35:09 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.629096053s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [........................................X...X...............] duration: 3.792393002s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:35:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.792415435s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.X............] duration: 6.756544488s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:34:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.756567551s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [..X.............X....X..............X....X..................] duration: 4.252709003s trace: https://jaeger.dev.storj.io/trace/4f7ff47c5b5081c9 FAIL (2026-05-26 11:34:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 26 11:18:42.420902978 (33h14m28.204243077s) [upload=31830]: Starting upload May 26 11:18:42.420951039 (33h14m28.204291138s) [upload=31830]: Done starting upload May 26 11:18:42.421028367 (33h14m28.204368466s) [upload=31830]: Uploading segments... May 26 11:18:42.421195004 (33h14m28.204535103s) [upload=31830]: Got next segment. Inline: false May 26 11:18:42.421224310 (33h14m28.204564399s) [upload=31830]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 26 11:18:42.421227987 (33h14m28.204568086s) [upload=31830]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 26 11:19:35.470954648 (33h15m21.254294747s) [upload=31830]: Next segment err: context deadline exceeded May 26 11:19:35.471205156 (33h15m21.254545255s) [upload=31830]: Done uploading segments... May 26 11:19:35.471236386 (33h15m21.254576475s) [upload=31830]: Upload finished. err: context deadline exceeded Duration: 4.252729062s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.........................................X...X..............] duration: 3.695756751s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:33:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.695772301s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.X...........] duration: 6.603377607s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:33:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.603397837s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..........................................X...X.............] duration: 3.6062305s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:33:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.606245359s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.X..........] duration: 6.738023588s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:33:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.738080635s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [...X.............X....X..............X....X.................] duration: 4.207606099s trace: https://jaeger.dev.storj.io/trace/4f7ff47c5b5081c9 FAIL (2026-05-26 11:32:59 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 26 11:18:42.420902978 (33h14m28.204243077s) [upload=31830]: Starting upload May 26 11:18:42.420951039 (33h14m28.204291138s) [upload=31830]: Done starting upload May 26 11:18:42.421028367 (33h14m28.204368466s) [upload=31830]: Uploading segments... May 26 11:18:42.421195004 (33h14m28.204535103s) [upload=31830]: Got next segment. Inline: false May 26 11:18:42.421224310 (33h14m28.204564399s) [upload=31830]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 26 11:18:42.421227987 (33h14m28.204568086s) [upload=31830]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 26 11:19:35.470954648 (33h15m21.254294747s) [upload=31830]: Next segment err: context deadline exceeded May 26 11:19:35.471205156 (33h15m21.254545255s) [upload=31830]: Done uploading segments... May 26 11:19:35.471236386 (33h15m21.254576475s) [upload=31830]: Upload finished. err: context deadline exceeded Duration: 4.20762797s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.X.........] duration: 6.547192284s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:32:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.54721721s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...........................................X...X............] duration: 3.808700721s trace: https://jaeger.dev.storj.io/trace/3f59f32ec8bcb666 FAIL (2026-05-26 11:32:20 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 3.808722202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 5 out of 60 is failed (threshold=3) [....X.............X....X..............X....X................] duration: 5.687266129s trace: https://jaeger.dev.storj.io/trace/4f7ff47c5b5081c9 FAIL (2026-05-26 11:32:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 26 11:18:42.420902978 (33h14m28.204243077s) [upload=31830]: Starting upload May 26 11:18:42.420951039 (33h14m28.204291138s) [upload=31830]: Done starting upload May 26 11:18:42.421028367 (33h14m28.204368466s) [upload=31830]: Uploading segments... May 26 11:18:42.421195004 (33h14m28.204535103s) [upload=31830]: Got next segment. Inline: false May 26 11:18:42.421224310 (33h14m28.204564399s) [upload=31830]: Flush batch item: pb.BatchRequestItem_ObjectBegin May 26 11:18:42.421227987 (33h14m28.204568086s) [upload=31830]: Flush batch item: pb.BatchRequestItem_SegmentBegin May 26 11:19:35.470954648 (33h15m21.254294747s) [upload=31830]: Next segment err: context deadline exceeded May 26 11:19:35.471205156 (33h15m21.254545255s) [upload=31830]: Done uploading segments... May 26 11:19:35.471236386 (33h15m21.254576475s) [upload=31830]: Upload finished. err: context deadline exceeded Duration: 5.687292437s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.X........] duration: 6.575272345s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:32:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.575314976s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.X.......] duration: 7.002230704s trace: https://jaeger.dev.storj.io/trace/1f80eefb3d29cbe3 FAIL (2026-05-26 11:31:45 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.002328481s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)

checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)