Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..........................X..............X..................] duration:
3.737218896s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:14:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.73724198s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..................................................X..X......] duration:
5.907243534s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:14:46 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 5.907259826s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [...........X.X..XX.X...............X..................X.....] duration:
6.603247063s trace:
https://jaeger.dev.storj.io/trace/4f326f7e35ea3a96
FAIL (2026-05-26 00:14:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.603269466s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...........................X..............X.................] duration:
3.677569154s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:14:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.677582178s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [............X.X..XX.X...............X..................X....] duration:
6.805407665s trace:
https://jaeger.dev.storj.io/trace/4f326f7e35ea3a96
FAIL (2026-05-26 00:14:06 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.805428535s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...................................................X..X.....] duration:
3.939198394s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:13:45 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 3.939219956s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.............X.X..XX.X...............X..................X...] duration:
6.521821021s trace:
https://jaeger.dev.storj.io/trace/4f326f7e35ea3a96
FAIL (2026-05-26 00:13:41 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.521844365s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [............................X..............X................] duration:
4.415193402s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:13:23 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.415223899s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....................................................X..X....] duration:
4.097702357s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:13:11 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 4.097738947s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [..............X.X..XX.X...............X..................X..] duration:
10.691584951s trace:
https://jaeger.dev.storj.io/trace/4f326f7e35ea3a96
FAIL (2026-05-26 00:13:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 10.691597825s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [...............X.X..XX.X...............X..................X.] duration:
36.884178118s trace:
https://jaeger.dev.storj.io/trace/4f326f7e35ea3a96
FAIL (2026-05-26 00:12:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 36.884186995s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................................................X..X...] duration:
4.002993879s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:12:20 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 4.003018927s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.............................X..............X...............] duration:
4.773030908s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:12:20 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.773049354s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [................X.X..XX.X...............X..................X] duration:
1m0.00134287s trace:
https://jaeger.dev.storj.io/trace/4f326f7e35ea3a96
FAIL (2026-05-26 00:12:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.0013541s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......................................................X..X..] duration:
4.082937476s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:11:46 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 4.082960058s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..............................X..............X..............] duration:
4.714727859s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:11:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.714744842s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.......................................................X..X.] duration:
4.222833798s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:11:22 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 4.222845922s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [........................................................X..X] duration:
1m0.002179525s trace:
https://jaeger.dev.storj.io/trace/54dd3f55a77d17ef
FAIL (2026-05-26 00:11:17 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320420647 (22h6m37.103760746s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320702816 (22h6m37.104042915s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.320993440 (22h6m37.104333539s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321256863 (22h6m37.104596962s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321827863 (22h6m37.105167962s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.321916452 (22h6m37.105256551s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322280286 (22h6m37.105620385s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322452585 (22h6m37.105792684s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.322950865 (22h6m37.106290964s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323536944 (22h6m37.106877043s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.323731265 (22h6m37.107071364s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324123954 (22h6m37.107464053s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.324414138 (22h6m37.107754227s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.325132458 (22h6m37.108472547s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:51.327055307 (22h6m37.110395406s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Uploading piece...
May 26 00:10:52.388936548 (22h6m38.172276647s) [upload=21163]: Next returned nil segment
May 26 00:10:52.388980332 (22h6m38.172320431s) [upload=21163]: Waiting for error group managing segments...
May 26 00:10:52.674703801 (22h6m38.4580439s) [noise=true , address=212.102.52.216:4399 , piece_id=K5B3VDSSOJYPKIQWSZA3LAIP6ECGX4UIYTLHJKLEV65EEROPAXWA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.676576304 (22h6m38.459916403s) [noise=true , address=212.102.52.214:4358 , piece_id=XYDUBDN62GKNQB3O5H2BWWROODBZKXNJVEZRXCYXTRE3XGTKPCZQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.685919140 (22h6m38.469259239s) [noise=true , address=212.102.52.215:4371 , piece_id=OJPOOTWXGORXQYWWFFOKOT33F6KJI4A2355TNSMYE4OQXRU3GM3Q, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.691062223 (22h6m38.474402322s) [noise=true , address=212.102.52.214:4347 , piece_id=OP75SIPBPG6SRO427YW5DXBI2LX3IGJYQ272EQABJCYXPJQSX3IA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.692733643 (22h6m38.476073742s) [noise=true , address=212.102.52.213:4341 , piece_id=IZ7SWWTG5TQT5L725QJU7SBWBYQ4HW2R6C4DO7C2LWEJL7UMKX3A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.695375413 (22h6m38.478715512s) [noise=true , address=212.102.52.213:4324 , piece_id=IHZCGYVL5XVHHQBTQGNXJKVDNCKGLMPPI7CMP5U5QSEC2KBKS2ZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.697960065 (22h6m38.481300164s) [noise=true , address=212.102.52.215:4390 , piece_id=E4XO5YC7J2INYSCZLMJO6VRV4DMDM4YDBYAOIHXR7NHCC5EZELKQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.699281385 (22h6m38.482621484s) [noise=true , address=212.102.52.216:4402 , piece_id=YIB3T4SMRTQJGTAVUESYSEKOEUGWYNSUUIE36NTNYCXJGBS6GZJA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.701163387 (22h6m38.484503486s) [noise=true , address=212.102.52.214:4367 , piece_id=P25PLZRCWMM5F2ZV2M6AE6KZH23PDQWYCC2KWJORUO55RG2BKAZA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.702857559 (22h6m38.486197658s) [noise=true , address=212.102.52.214:4353 , piece_id=WZG7PJO5RODWWW63G26AKA5ZAP4LUVF66AOOKRIIBUUO4IWP7GOA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.705263059 (22h6m38.488603159s) [noise=true , address=212.102.52.215:4386 , piece_id=S5YIYIWUEMKONHJACR7E7PO7U7N2FMU3GH3OO4ULB77LLLC624MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707419504 (22h6m38.490759593s) [noise=true , address=212.102.52.217:4433 , piece_id=IBUBF2KSK4XJJY3GGUU7ELHZA6QS7LENZYUKXAA24NYSSAYJQAXQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.707786013 (22h6m38.491126112s) [noise=true , address=212.102.52.217:4420 , piece_id=JDI5NNY6OJUU2DSIFYDTDVYJD2POJDFPAD3HA237DZ43E2R2MYHQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.709872454 (22h6m38.493212543s) [noise=true , address=212.102.52.216:4407 , piece_id=DWZNVQEGJ2V34UDSVXLGGBHLKVCF4QYTWLYJ5GWUUVWGVUQCDSVA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.730564756 (22h6m38.513904845s) [noise=true , address=212.102.52.215:4382 , piece_id=BAJ5RC7425LFHND3DAXX4YSESMOUIRZOH2SHWHITX4KT3OAE54YQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736735980 (22h6m38.520076079s) [noise=true , address=212.102.52.217:4431 , piece_id=P7SKOTYTIASYYKMEFKYQTDJ3ZP2CVYOYATQPEYQDG2P65ZUOPWQA, seg_pos={0 0}, upload=21163]: Done uploading piece. err: <nil>
May 26 00:10:52.736752663 (22h6m38.520092762s) [seg_pos={0 0}, upload=21163]: Satisfied cohort requirements after 16 pieces
May 26 00:10:52.736921284 (22h6m38.520261383s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f4fqtM26XZspG77Sjho67Tx9ucw1hAwg98xbPxqUcE6f1nH5U): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.736929240 (22h6m38.520269339s) [noise=true , address=212.102.52.217:4424 , piece_id=56YU26BQTQY7FJUHZ7T5ISAPEYY7QAJI2Q3Z6J2MDAKYBVYACJ7A, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737040080 (22h6m38.520380179s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1SUcotB1144nMnt3JXXbVyNoz9aS6352uH9cGptt4RqKbvQkRv): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737054549 (22h6m38.520394678s) [noise=true , address=212.102.52.213:4339 , piece_id=MN3JN37HKGKLVKJYXMXVSMITKU4PCPH6NDD7Z5U37YZ62ZS2U4TQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737102950 (22h6m38.520443049s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1272rh1tzNBuEmXqx4MpBe7NKXPac2wuNPCEpYqFk56gTrodcU2): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737108672 (22h6m38.520448771s) [noise=true , address=212.102.52.216:4405 , piece_id=OXJTIEZ2HUNIGFLRCEM64MELMI75XZUXRMQLM4EYFIWZWQIXN7MQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737196889 (22h6m38.520536988s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12f1unaJrBkG1zw5XdBmCMeS8EKdqELYbQcr5cLq513xvctxcED): protocol: expected piece hash; context canceled; EOF
May 26 00:10:52.737201459 (22h6m38.520541558s) [noise=true , address=212.102.52.213:4322 , piece_id=KHVYCOXFDTYTYWSGQBU65FZBVLULQZW25PXAXDTGVDBCREPSKZUQ, seg_pos={0 0}, upload=21163]: Long tail cancellation due to optimal threshold
May 26 00:10:52.737216567 (22h6m38.520556666s) [seg_pos={0 0}, upload=21163]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 26 00:10:52.737295508 (22h6m38.520635597s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 26 00:10:52.737299646 (22h6m38.520639745s) [upload=21163]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 26 00:11:00.530433246 (22h6m46.313773355s) [upload=21163]: Done uploading segments...
May 26 00:11:00.530455227 (22h6m46.313795347s) [upload=21163]: Upload finished. err: <nil>
Duration: 1m0.002199683s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...............................X..............X.............] duration:
3.785969901s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:11:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.785989969s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [.................X.X..XX.X...............X..................] duration:
6.592712805s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:10:47 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.592729968s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [..................X.X..XX.X...............X.................] duration:
6.777377461s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:10:21 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.777414771s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [...................X.X..XX.X...............X................] duration:
50.467078828s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:10:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 50.467094087s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
4.139041293s trace:
https://jaeger.dev.storj.io/trace/24b77a416611287
FAIL (2026-05-26 00:10:01 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 26 00:08:24.368303179 (22h4m10.151643328s) [upload=21127]: Starting upload May 26 00:08:24.368373913 (22h4m10.151714082s) [upload=21127]: Done starting upload May 26 00:08:24.368507228 (22h4m10.151847357s) [upload=21127]: Uploading segments... May 26 00:08:24.368773056 (22h4m10.152113165s) [upload=21127]: Got next segment. Inline: false May 26 00:08:24.368815546 (22h4m10.152155655s) [upload=21127]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 26 00:08:24.368821718 (22h4m10.152161827s) [upload=21127]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 26 00:08:51.598659622 (22h4m37.381999721s) [upload=21127]: Next segment err: context deadline exceeded May 26 00:08:51.598886114 (22h4m37.382226213s) [upload=21127]: Done uploading segments... May 26 00:08:51.598997626 (22h4m37.382337736s) [upload=21127]: Upload finished. err: context deadline exceeded
Duration: 4.139065481s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [................................X..............X............] duration:
4.060268411s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:09:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.060290252s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.................................X..............X...........] duration:
3.704765997s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:09:29 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.70477816s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..................................X..............X..........] duration:
4.829965111s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:09:12 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.829988644s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
6 out of 60 is failed (threshold=3) [....................X.X..XX.X...............X...............] duration:
29.535609961s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:09:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.53563004s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [X.........................................................X.] duration:
3.976652222s trace:
https://jaeger.dev.storj.io/trace/24b77a416611287
FAIL (2026-05-26 00:09:03 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 26 00:08:24.368303179 (22h4m10.151643328s) [upload=21127]: Starting upload May 26 00:08:24.368373913 (22h4m10.151714082s) [upload=21127]: Done starting upload May 26 00:08:24.368507228 (22h4m10.151847357s) [upload=21127]: Uploading segments... May 26 00:08:24.368773056 (22h4m10.152113165s) [upload=21127]: Got next segment. Inline: false May 26 00:08:24.368815546 (22h4m10.152155655s) [upload=21127]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 26 00:08:24.368821718 (22h4m10.152161827s) [upload=21127]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 26 00:08:51.598659622 (22h4m37.381999721s) [upload=21127]: Next segment err: context deadline exceeded May 26 00:08:51.598886114 (22h4m37.382226213s) [upload=21127]: Done uploading segments... May 26 00:08:51.598997626 (22h4m37.382337736s) [upload=21127]: Upload finished. err: context deadline exceeded
Duration: 3.976696786s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.X.........................................................X] duration:
1m0.001089158s trace:
https://jaeger.dev.storj.io/trace/24b77a416611287
FAIL (2026-05-26 00:08:51 UTC)
orbiter warning: operation: sync2.Copy ======ERROR====== uplink: encryption: context deadline exceeded ======LOGS====== May 26 00:08:24.368303179 (22h4m10.151643328s) [upload=21127]: Starting upload May 26 00:08:24.368373913 (22h4m10.151714082s) [upload=21127]: Done starting upload May 26 00:08:24.368507228 (22h4m10.151847357s) [upload=21127]: Uploading segments... May 26 00:08:24.368773056 (22h4m10.152113165s) [upload=21127]: Got next segment. Inline: false May 26 00:08:24.368815546 (22h4m10.152155655s) [upload=21127]: Flush batch item: *pb.BatchRequestItem_ObjectBegin May 26 00:08:24.368821718 (22h4m10.152161827s) [upload=21127]: Flush batch item: *pb.BatchRequestItem_SegmentBegin May 26 00:08:51.598659622 (22h4m37.381999721s) [upload=21127]: Next segment err: context deadline exceeded May 26 00:08:51.598886114 (22h4m37.382226213s) [upload=21127]: Done uploading segments... May 26 00:08:51.598997626 (22h4m37.382337736s) [upload=21127]: Upload finished. err: context deadline exceeded
Duration: 1m0.001105671s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [X....................X.X..XX.X...............X..............] duration:
6.471563235s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:08:31 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.471580178s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [...................................X..............X.........] duration:
3.895927539s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:08:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.895949872s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
7 out of 60 is failed (threshold=3) [.X....................X.X..XX.X...............X.............] duration:
6.97503886s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:07:45 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.975060862s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [....................................X..............X........] duration:
3.732574163s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:07:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.732594752s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.....................................X..............X.......] duration:
4.012738083s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:06:57 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.012754334s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [X.X....................X.X..XX.X...............X............] duration:
6.650898103s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:06:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.65091773s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..X.........................................................] duration:
4.537314855s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-26 00:06:43 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.537337808s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...X........................................................] duration:
3.886524116s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-26 00:06:24 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 3.886542332s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [.X.X....................X.X..XX.X...............X...........] duration:
6.485408951s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:06:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.48541883s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [......................................X..............X......] duration:
4.54354467s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:06:09 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.543566762s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....X.......................................................] duration:
5.145003942s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-26 00:05:54 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 5.145025555s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.......................................X..............X.....] duration:
4.205867031s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:05:36 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.20589296s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....X......................................................] duration:
6.986623347s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-26 00:05:15 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 6.986642854s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
8 out of 60 is failed (threshold=3) [..X.X....................X.X..XX.X...............X..........] duration:
6.45707713s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:05:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.457091076s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......X.....................................................] duration:
4.09723038s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-26 00:04:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 4.097258093s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......X....................................................] duration:
42.287512104s trace:
https://jaeger.dev.storj.io/trace/154f04129c3ba223
FAIL (2026-05-26 00:04:30 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
H5UA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.669978445 (21h21m58.453318544s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670570324 (21h21m58.453910423s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.670977932 (21h21m58.454318031s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671082672 (21h21m58.454422771s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671399046 (21h21m58.454739145s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671505018 (21h21m58.454845117s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671768601 (21h21m58.4551087s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.671873380 (21h21m58.455213479s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672172411 (21h21m58.45551251s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672225192 (21h21m58.455565291s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672604115 (21h21m58.455944214s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672647878 (21h21m58.455987977s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.672979411 (21h21m58.45631951s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673026311 (21h21m58.45636641s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:12.673319009 (21h21m58.456659108s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Uploading piece...
May 25 23:26:13.747141337 (21h21m59.530481436s) [upload=20458]: Next returned nil segment
May 25 23:26:13.747155513 (21h21m59.530495612s) [upload=20458]: Waiting for error group managing segments...
May 25 23:26:14.031706029 (21h21m59.815046128s) [noise=true , address=212.102.52.213:4334 , piece_id=TMEOBZ5ZZQRVYSI2MX47UN3WZCXPTNO466DTWCN4FELUUN2VEYDA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034098815 (21h21m59.817438914s) [noise=true , address=212.102.52.216:4409 , piece_id=D65YC4OFDAUR2PRJHXPJNML7HI6XO3RURGIHDHZUF5KS5QTJO2TQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034662339 (21h21m59.818002439s) [noise=true , address=212.102.52.214:4362 , piece_id=ZSAFYLTPN3ESF7KDM6RXVTPIHI2O5BJ77WJQXDRGXEIM5IUTEVMQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.034708708 (21h21m59.818048807s) [noise=true , address=212.102.52.217:4419 , piece_id=3IAS5VH7QJ2SJOMUCY2B4MCNYECWLHUH3AGH6MOGM4NS6KDAH5UA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035071280 (21h21m59.818411379s) [noise=true , address=212.102.52.217:4424 , piece_id=OHAYPIY666MWXWJTMWLPWTPE3RAXI6SG6NRFZRQS7VO4GKYCOECQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.035492163 (21h21m59.818832272s) [noise=true , address=212.102.52.217:4435 , piece_id=52SDHCU6EYU7XE5P26JBEBD3WDBZJS2K7T4UEITQY76LYQ7CWOFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.036495778 (21h21m59.819835877s) [noise=true , address=212.102.52.215:4369 , piece_id=7U5BJDB4RNP2VJWT5LMMI37B2FARBL6FB4TNTQ4BYCHOJ6XPNILA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.037130078 (21h21m59.820470177s) [noise=true , address=212.102.52.214:4353 , piece_id=PGJXAO27LI563LSACTWLNGDDTLWPF2KOKSF6YBZY7YGPMD7C66IQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.040472716 (21h21m59.823812815s) [noise=true , address=212.102.52.216:4396 , piece_id=MM46S6QCH6X7LEUCBZWF3LM4C7C45WV6OEUY5CVVJ7QPTJJMC4VA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.042158933 (21h21m59.825499032s) [noise=true , address=212.102.52.214:4365 , piece_id=WLGBUXLDUT2BVURAHW7EN3ZBRE7AAAE3NO4LHWGT2DPUUYXR5M5Q, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045074927 (21h21m59.828415026s) [noise=true , address=212.102.52.217:4439 , piece_id=SI3MH3CMRPJOJVUGU2KZBW6LTE7OBTGM5QZ6NG5J77DOXBJYLDHA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.045309063 (21h21m59.828649162s) [noise=true , address=212.102.52.215:4384 , piece_id=OIWVKV3EPC5OTSU36PSFDMKIBUM63VL3ILYIJUNC6VIK4BBP4IMA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046163785 (21h21m59.829503874s) [noise=true , address=212.102.52.214:4346 , piece_id=2LVJNXGQ27JYREAYEHZOAOJCJMZ4P7Q3OHVJ6SMR2BPXTZW25UCQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.046705198 (21h21m59.830045297s) [noise=true , address=212.102.52.213:4328 , piece_id=BJMKGMFRF7TC3UJDBYPDGQRR2I5T23XQIHZIESYTTRYNAYCYXNVA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.062034446 (21h21m59.845374546s) [noise=true , address=212.102.52.213:4343 , piece_id=IUD6WGNSWDSRXWNS5UBGQA7AS4EQO5M2V4C7WXNEHMJVKOZADU3A, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070397062 (21h21m59.853737161s) [noise=true , address=212.102.52.213:4322 , piece_id=TW24WKT2HYX65MOG6ILC2GYJPHYS5VRAEKPWGNF3MMPVJYMAZPHQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: <nil>
May 25 23:26:14.070415798 (21h21m59.853755897s) [seg_pos={0 0}, upload=20458]: Satisfied cohort requirements after 16 pieces
May 25 23:26:14.070581143 (21h21m59.853921242s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VK5hn1TRuMk5ZXTat7nmjx4kXZec3RDyQ89iQrrrRzVh2xPVF): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070588727 (21h21m59.853928826s) [noise=true , address=212.102.52.215:4388 , piece_id=34VBVPZJSEKHER4HKH4X4TESNLCPDWNBP6GH4ICDSDWHBAEX3AEQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070817384 (21h21m59.854157483s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12cCanaWd2hWD8vcQCophdGPopVihkjKSuUcB7ZyqqBcK1JTvSf): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070822854 (21h21m59.854162943s) [noise=true , address=212.102.52.216:4411 , piece_id=PAKM6TQQJ2A5M2PO5REFGZ4EVXTK67QXCANGMCVADZKRPC6VHBFQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.070908688 (21h21m59.854248787s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YyuYGM7ttMRXybutoxVQBktQPes1XMce5fTs4BUfQVykeZtqT): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.070913157 (21h21m59.854253256s) [noise=true , address=212.102.52.216:4397 , piece_id=RERO53Z3UUNEKZ47V7VN7SSU3LR5OFVEOFFSEZDYKL7NECDBHYWQ, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071162232 (21h21m59.854502341s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12BL6sUUowTDCFa3Zu7Xv5SheJRFGzEHVd1yq3goxmvT5WggLg4): protocol: expected piece hash; context canceled; EOF
May 25 23:26:14.071169757 (21h21m59.854509866s) [noise=true , address=212.102.52.215:4378 , piece_id=76YH63TUZZXDVYCO3VZAWO4WTBBAEXV4V2F53TYJWGP6AGOESNQA, seg_pos={0 0}, upload=20458]: Long tail cancellation due to optimal threshold
May 25 23:26:14.071187790 (21h21m59.854527889s) [seg_pos={0 0}, upload=20458]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 23:26:14.071292471 (21h21m59.85463257s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 23:26:14.071297069 (21h21m59.854637168s) [upload=20458]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 23:26:38.384275645 (21h22m24.167615734s) [upload=20458]: Done uploading segments...
May 25 23:26:38.384290073 (21h22m24.167630172s) [upload=20458]: Upload finished. err: <nil>
Duration: 42.287534137s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [........................................X..............X....] duration:
4.28144786s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:04:25 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 4.281461846s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [X..X.X....................X.X..XX.X...............X.........] duration:
6.652953596s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:04:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.652972612s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.........................................X..............X...] duration:
3.62306247s trace:
https://jaeger.dev.storj.io/trace/3bd7a1328937f878
FAIL (2026-05-26 00:03:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 3.623133105s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [.X..X.X....................X.X..XX.X...............X........] duration:
6.471670446s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:03:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.471694232s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
9 out of 60 is failed (threshold=3) [..X..X.X....................X.X..XX.X...............X.......] duration:
6.36459607s trace:
https://jaeger.dev.storj.io/trace/538e5c94a3d33f13
FAIL (2026-05-26 00:03:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.364614343s
core
uplink
us-select-1
us1-placement-20
|