Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.........X.........X.............................X..........] duration:
3.861917562s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:18:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.861937679s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................X.........................] duration:
4.426910184s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:18:40 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.426927466s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [....X..X..X.........................X..X..XXX.XXX.........XX] duration:
1m0.000947086s trace:
https://jaeger.dev.storj.io/trace/1e14077bb6a31bef
FAIL (2026-05-25 22:18:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000953959s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.............X..............................................] duration:
4.669180243s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:18:06 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.669201442s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [..........X.........X.............................X.........] duration:
3.770725973s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:17:52 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.770742365s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................X........................] duration:
4.017177412s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:17:46 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.017201897s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..............X.............................................] duration:
4.569668841s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:17:22 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.569684121s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [.....X..X..X.........................X..X..XXX.XXX.........X] duration:
1m0.001596412s trace:
https://jaeger.dev.storj.io/trace/3a445ad942fde157
FAIL (2026-05-25 22:17:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001615968s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................X.......................] duration:
4.033323761s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:17:03 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.033347205s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [...........X.........X.............................X........] duration:
3.762013978s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:16:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.762027344s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...............X............................................] duration:
4.849336936s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:16:38 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.849361642s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................X......................] duration:
4.021526709s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:16:20 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.02154228s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [......X..X..X.........................X..X..XXX.XXX.........] duration:
20.074282765s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:16:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.074303764s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
4.923883495s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:16:09 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.923904055s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................X.....................] duration:
4.167280762s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:15:37 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.167303837s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [............X.........X.............................X.......] duration:
3.844376752s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:15:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.844397481s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.......X..X..X.........................X..X..XXX.XXX........] duration:
6.627701148s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:15:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.627721929s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
5.30239685s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:15:03 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.302418121s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................X....................] duration:
4.169031462s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:14:49 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.169057362s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [........X..X..X.........................X..X..XXX.XXX.......] duration:
18.918698924s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:14:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.918723562s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.............X.........X.............................X......] duration:
3.931622613s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:14:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.931646499s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
5.707397119s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:14:02 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.707420473s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.........X..X..X.........................X..X..XXX.XXX......] duration:
6.450318159s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:14:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.450361701s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................X...................] duration:
5.224086326s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:13:55 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 5.224109409s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [..........X..X..X.........................X..X..XXX.XXX.....] duration:
29.784773541s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:13:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.78479357s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [..............X.........X.............................X.....] duration:
3.661586488s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:13:38 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.661630953s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [...........X..X..X.........................X..X..XXX.XXX....] duration:
19.997592672s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:12:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 19.997658097s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................X..................] duration:
6.796172047s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:12:55 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 6.796188196s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
5.933338479s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:12:49 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.933361723s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [...............X.........X.............................X....] duration:
4.031061203s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:12:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.031078797s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
4.563909315s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:12:27 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.563930377s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [................X.........X.............................X...] duration:
3.655691866s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:12:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.655717414s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
5.707668167s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:12:09 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 5.707679298s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
4.958655384s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:12:07 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 4.958677537s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
3.953058181s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:11:41 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 3.95306839s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
29.529024202s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:11:37 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 29.529046194s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [.................X.........X.............................X..] duration:
3.724815002s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:11:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.724839308s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [............X..X..X.........................X..X..XXX.XXX...] duration:
6.902327736s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:11:34 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.902349287s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
5.578832068s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:11:03 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.57885951s
core
uplink
eu1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [.............X..X..X.........................X..X..XXX.XXX..] duration:
20.847085731s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:11:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.847109487s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [..................X.........X.............................X.] duration:
4.275519146s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:10:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.27554238s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [..............X..X..X.........................X..X..XXX.XXX.] duration:
56.18185776s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:10:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 56.181877627s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
5.559733952s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:10:25 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.559754632s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
3.996609375s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:10:02 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 3.996631579s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
3 out of 60 is failed (threshold=6) [...................X.........X.............................X] duration:
5.000927413s trace:
https://jaeger.dev.storj.io/trace/3d3b2dcb253ea4ee
FAIL (2026-05-25 22:09:59 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 5.000937202s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
4.205564909s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:09:44 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.205587634s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [...............X..X..X.........................X..X..XXX.XXX] duration:
1m0.001177641s trace:
https://jaeger.dev.storj.io/trace/295f167ae78b8a17
FAIL (2026-05-25 22:09:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001187359s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
4.166451225s trace:
https://jaeger.dev.storj.io/trace/5c964271d20d5817
FAIL (2026-05-25 22:09:24 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PZBYA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016526591 (19h56m17.79986668s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.016942624 (19h56m17.800282723s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017345213 (19h56m17.800685312s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017704899 (19h56m17.801044998s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.017945076 (19h56m17.801285175s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018092017 (19h56m17.801432116s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018464699 (19h56m17.801804798s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018555913 (19h56m17.801896012s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.018953491 (19h56m17.80229359s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019244447 (19h56m17.802584546s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019258222 (19h56m17.802598321s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019539188 (19h56m17.802879287s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.019830494 (19h56m17.803170593s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020110528 (19h56m17.803450627s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:32.020441400 (19h56m17.803781499s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Uploading piece...
May 25 22:00:33.078323480 (19h56m18.861663579s) [upload=19092]: Next returned nil segment
May 25 22:00:33.078333228 (19h56m18.861673327s) [upload=19092]: Waiting for error group managing segments...
May 25 22:00:33.362737910 (19h56m19.146078009s) [noise=true , address=212.102.52.217:4435 , piece_id=UGNSVT6BMSSG4ONLIT224QXGLAMVTDTFYOBI2PTRW3NRT72EKUTA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.366505599 (19h56m19.149845698s) [noise=true , address=212.102.52.216:4398 , piece_id=PJVDU2MLBCOCPAX5ZY6UZU2A6LPOBA6BK3ESUO2YAHPUD5AHTVJA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369025236 (19h56m19.152365335s) [noise=true , address=212.102.52.215:4391 , piece_id=BIUJXSMRFAYOATOIVZ73WR236BLHIQWEYWOZZ6MFPOICNG5HJH3A, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.369736344 (19h56m19.153076443s) [noise=true , address=212.102.52.213:4324 , piece_id=PEYXUGDB5LZ5X6ZL3BFW5Y7KVHAX4XKK3SOZZ4NIF7P5DMSLNY7Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373386979 (19h56m19.156727128s) [noise=true , address=212.102.52.214:4357 , piece_id=W7O3PGV2WDB3ZRMNAXZCDIE7JJJCIBK7YRRU643FR33EXYZY4KQA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.373749210 (19h56m19.15708931s) [noise=true , address=212.102.52.213:4336 , piece_id=LYOKI76BSM6VOD6BQ4LNX6EFGIUMSSHC2IEC5CIA52N52J7ZTO2Q, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.376611661 (19h56m19.159951792s) [noise=true , address=212.102.52.213:4340 , piece_id=3U3H5IRALM7QW6A4FV2EQNPFSVT33GXJL7NYOJIW6A5T3LBZPBVQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385293187 (19h56m19.168633286s) [noise=true , address=212.102.52.217:4417 , piece_id=SNP2IHUE55IHBBPUJUGKOABHKNZ3EAQNPERP4CM6H5MNM4ZNWHUA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.385441840 (19h56m19.16878196s) [noise=true , address=212.102.52.214:4365 , piece_id=IIR5IJNAE76RYZ6YWVZ4Z67ZGKENSNRHDPNRVO3PDJXQC6YPZBYA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386502284 (19h56m19.169842383s) [noise=true , address=212.102.52.217:4420 , piece_id=LKY4ZOYCZICHV5WT5QLA6UZQ2N5JLX32CASLNHCBVV7PRV2IHRZQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.386550555 (19h56m19.169890654s) [noise=true , address=212.102.52.216:4408 , piece_id=V365XJO35CB2AT45TJTMIL7R7B42XY3GQIG5TVE3GYXAWWMXKQLQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.387214632 (19h56m19.170554731s) [noise=true , address=212.102.52.214:4349 , piece_id=E4MU3EUSQ3MTS6THBGHYE2NP2TL42WH677WVCLZZRNCIHSMJF7GA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.389717318 (19h56m19.173057417s) [noise=true , address=212.102.52.215:4374 , piece_id=Z4JJUIXBZN43BLG57VKZQV2QNTAY5D754GMSFEHVSSNXSFNYSAKA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391350674 (19h56m19.174690773s) [noise=true , address=212.102.52.214:4350 , piece_id=47OQ2QSA54PSPA6SSP7U6YWMKHNV3I2XDDGUCI73V6MLWKOTIWIA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.391666748 (19h56m19.175006847s) [noise=true , address=212.102.52.217:4430 , piece_id=RIKS76WYCYE4AXJDZDBVHSCIHDWDRWH6WWMNAAQGJVSJQIEGMZEQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392934657 (19h56m19.176274756s) [noise=true , address=212.102.52.215:4376 , piece_id=IBG6TFEQKRLK56QA7Z4X5TYMQSOXHQ27CJXVKIP2FCYBQZW6DZLA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: <nil>
May 25 22:00:33.392943844 (19h56m19.176283933s) [seg_pos={0 0}, upload=19092]: Satisfied cohort requirements after 16 pieces
May 25 22:00:33.393089582 (19h56m19.176429682s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ohfhnBX7MWrCCdmn8rKNQjksDizY9K7ar6Htz3BVTrsPPkMYn): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393096245 (19h56m19.176436334s) [noise=true , address=212.102.52.213:4344 , piece_id=GNBJGA4GCWLWBGI2K7THTZGVG35TDSCGRHC5UTT3MNRPSHBQX7PA, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393196536 (19h56m19.176536625s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Px3pe9nYbzRQ9tb2RDpL3ZGKViv6rTTm8AvhVcofGfrUWg5bk): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393202889 (19h56m19.176542988s) [noise=true , address=212.102.52.216:4409 , piece_id=MZEXBOHGKTUTAGNCZAPFSL4ZIWN32NI4NTF6DTL7JCI4DW7TPYSQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393278763 (19h56m19.176618862s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i3YC1eLBhg5PiyPZhHFwFLprKxqgwo37862bYcYNsrB8rLdUG): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393288602 (19h56m19.176628701s) [noise=true , address=212.102.52.215:4382 , piece_id=G2FUEPQG5VITPPR2VTA4XC6XRAPLADKO44JJUMU2XZ7POWLDC6QQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393314181 (19h56m19.17665428s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 22:00:33.393319532 (19h56m19.176659621s) [noise=true , address=212.102.52.216:4416 , piece_id=XND2VKBR3BBR56NGITX7CMIEMTC7HZ5S6Q2DBSRLY5DV6JKSQQGQ, seg_pos={0 0}, upload=19092]: Long tail cancellation due to optimal threshold
May 25 22:00:33.393341573 (19h56m19.176681662s) [seg_pos={0 0}, upload=19092]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 22:00:33.393400305 (19h56m19.176740444s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 22:00:33.393406276 (19h56m19.176746395s) [upload=19092]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 22:00:53.238873899 (19h56m39.022213998s) [upload=19092]: Done uploading segments...
May 25 22:00:53.238893697 (19h56m39.022233796s) [upload=19092]: Upload finished. err: <nil>
Duration: 4.166470883s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
5.308341927s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:09:22 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.308363708s
core
uplink
eu1
|
checkers.Uplink{Satellite: "12L9ZFwhzVpuEKMUNUqkaTLGzwY9G24tbiigLiXpmZWKwmcNDDs@eu1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.eu1.remote)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
5.596593056s trace:
https://jaeger.dev.storj.io/trace/67f9a81c0292293f
FAIL (2026-05-25 22:09:07 UTC)
orbiter warning: operation: StatBucket uplink: bucket: metaclient: context deadline exceeded
Duration: 5.596615509s
core
uplink
eu1
|