Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................X.................] duration:
4.013265017s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:13:34 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.013290316s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...XX........................X...........X..................] duration:
20.446849982s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:13:04 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.446902662s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................X................] duration:
4.159238175s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:12:23 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.159254276s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [............................................X...............] duration:
5.88434425s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:12:02 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 5.884367925s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [....XX........................X...........X.................] duration:
6.937088122s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:11:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.937110785s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.....XX........................X...........X................] duration:
6.970283359s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:11:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.970310511s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.............................................X..............] duration:
7.880716793s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:11:09 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 7.880746089s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [......XX........................X...........X...............] duration:
7.215329968s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:10:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.215351629s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..............................................X.............] duration:
4.278883815s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:10:33 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.278914374s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.......XX........................X...........X..............] duration:
6.743365808s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:10:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.74339368s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...............................................X............] duration:
3.969114325s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:10:00 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 3.969235896s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........XX........................X...........X.............] duration:
11.300414706s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:09:18 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.300437579s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.........XX........................X...........X............] duration:
18.378167501s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:09:07 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 18.378193832s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [................................................X...........] duration:
4.686414583s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:08:50 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.686438207s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..........XX........................X...........X...........] duration:
29.751831699s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:08:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 29.751858489s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.................................................X..........] duration:
6.75896499s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:08:18 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 6.758978575s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..................................................X.........] duration:
5.499820845s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:07:57 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 5.499864998s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...........XX........................X...........X..........] duration:
8.809400082s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:07:11 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 8.809418005s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...................................................X........] duration:
4.230439857s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:06:59 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.230480875s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [....................................................X.......] duration:
4.160983571s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:06:09 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.161010171s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [............XX........................X...........X.........] duration:
6.549841186s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:06:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.549884277s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.....................................................X......] duration:
8.840542345s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:05:28 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 8.840574946s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.............XX........................X...........X........] duration:
15.467910422s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:05:16 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 15.467933936s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [......................................................X.....] duration:
5.184150559s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:04:12 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 5.184172961s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..............XX........................X...........X.......] duration:
6.876029223s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:04:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.87605939s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.......................................................X....] duration:
5.935994689s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:03:42 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 5.936034234s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...............XX........................X...........X......] duration:
6.998910149s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:02:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.998953141s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [................XX........................X...........X.....] duration:
12.801748914s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:02:40 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 12.801773291s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [........................................................X...] duration:
3.997663862s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:02:32 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 3.997686825s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.................XX........................X...........X....] duration:
11.696876519s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:02:13 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 11.696909913s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..................XX........................X...........X...] duration:
6.672038792s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:01:48 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.672061465s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [.........................................................X..] duration:
4.015821649s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:01:35 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 4.015839493s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [..........................................................X.] duration:
3.965306544s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:01:11 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 3.965319327s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
1 out of 60 is failed (threshold=3) [...........................................................X] duration:
1m0.002529735s trace:
https://jaeger.dev.storj.io/trace/4dbec51c2511c446
FAIL (2026-05-24 17:01:07 UTC)
orbiter warning:
operation: DeleteObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
8.437055023s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.917400970 (574h12m18.438224715s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.918564570 (574h12m18.439388325s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.919783306 (574h12m18.440607061s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.920955152 (574h12m18.441778897s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.922161203 (574h12m18.442984968s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.923526478 (574h12m18.444350223s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.924759090 (574h12m18.445582835s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.925870040 (574h12m18.446693785s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.927047337 (574h12m18.447871082s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.928363307 (574h12m18.449187052s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.929573235 (574h12m18.450396991s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.930852096 (574h12m18.451675841s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.932068537 (574h12m18.452892282s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:07.933235203 (574h12m18.454058948s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Uploading piece...
May 24 17:00:09.006228623 (574h12m19.527052378s) [upload=550742]: Next returned nil segment
May 24 17:00:09.006244474 (574h12m19.527068229s) [upload=550742]: Waiting for error group managing segments...
May 24 17:00:09.289444894 (574h12m19.810268639s) [noise=true , address=212.102.52.213:4326 , piece_id=EOJW5RI6BICW3PV23MJQL724UN2G5YYLBWPAYCUS3EBO2JTHSJYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290196047 (574h12m19.811019812s) [noise=true , address=212.102.52.215:4370 , piece_id=R5EF4DUUPSRQO4ARGO4CWKJWYFBK5B4MXANM5LSATW5E4EHW2XHA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.290263135 (574h12m19.8110869s) [noise=true , address=212.102.52.216:4414 , piece_id=HEISAHE5WVNKBMRLZQJ47IWGZFTFFVK53GTFI2OPYC7C4WFLB44Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291050016 (574h12m19.811873771s) [noise=true , address=212.102.52.215:4372 , piece_id=FTRV34BNHAAQTE37LGQGODQWMBCCV5UYY7ZABCNAPS7C2G4X2M4A, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291252433 (574h12m19.812076198s) [noise=true , address=212.102.52.213:4325 , piece_id=7FZQKJVPU7CNCWN6VAW2CUVBXHNUBDK5OJDKWSHHXPGN3IJCPRMQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.291594485 (574h12m19.81241825s) [noise=true , address=212.102.52.215:4390 , piece_id=DK6VEDKZPKQW4VPVEPQBLU2XGXUUB4LU7R6RMDEICFLCTKYHOYQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.292235157 (574h12m19.813058912s) [noise=true , address=212.102.52.213:4339 , piece_id=KXKNMM6B4ZGLWWZWVB6ZJIXHRF4PJLO7S76NUTIPAHZNE75HJ73Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.293306662 (574h12m19.814130428s) [noise=true , address=212.102.52.216:4412 , piece_id=VYVEDSSQLTYY2LB7I2263UN7CFCLVJFRHVXCZBM5O4C6SUDBLMRQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.294913228 (574h12m19.815736983s) [noise=true , address=212.102.52.216:4404 , piece_id=SQKMLLSNXJMQRK5AIFUYTPBPZBV77N6YC7E7K3A3NSG4P56LSOBQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.298879585 (574h12m19.81970333s) [noise=true , address=212.102.52.217:4420 , piece_id=H3F24PDKPKG6KZP7LDUUBXJ2EWFJESIOGCGJT2W2UUSP7ODI6QXQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299027668 (574h12m19.819851413s) [noise=true , address=212.102.52.214:4345 , piece_id=MXJV5EEK3BEDWUAWZMC6CQ2JXHNUWHYGG7AGBAN5IXQEWSLS43DQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.299516592 (574h12m19.820340337s) [noise=true , address=212.102.52.216:4408 , piece_id=2XU4MRKGDB6X5WZUUYWYAYJ6PKMSCWWW655OEX4HNGJL3BEJ3GVA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.300511700 (574h12m19.821335445s) [noise=true , address=212.102.52.213:4340 , piece_id=U37J5DRZLFZ7TUV7LGNKOHLE27NSAEBO7JKJHDI5XL2GU2VHVGPA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.301548899 (574h12m19.822372654s) [noise=true , address=212.102.52.217:4437 , piece_id=5WX5UKGNPLY5OH6U5HRSP6XFYWHWHQTZW45KD3W3VXW3ROQBMZDA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.303632485 (574h12m19.82445624s) [noise=true , address=212.102.52.215:4369 , piece_id=JUFX4KYPYZKILKS6FZSI4TGNRCAS6KCRSTRPJ27FTQJYPMTARX4Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304121798 (574h12m19.824945543s) [noise=true , address=212.102.52.214:4357 , piece_id=TB5ZRYHHM7V2ARMV65K7OVTPWW6DDDPPA5DLMJGXLWBUDI2D3Z5Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: <nil>
May 24 17:00:09.304134162 (574h12m19.824957907s) [seg_pos={0 0}, upload=550742]: Satisfied cohort requirements after 16 pieces
May 24 17:00:09.304365533 (574h12m19.825189288s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12APEgwRugTmZkJ9dXZ71QKCqMZVfYNX7PA8FJFSS5Lx5TGfHpU): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304376445 (574h12m19.82520019s) [noise=true , address=212.102.52.214:4349 , piece_id=CKCBIARJYRJ6J2CW4WIYEK2FQNSDMNVR2KU3HZPFSQUBXVVIVKQA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304388327 (574h12m19.825212072s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mHkZ2yrJaVP3EbaU7YcSCzhcLHCJ58mcYYJpCamzs6oCBErfp): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304393817 (574h12m19.825217572s) [noise=true , address=212.102.52.217:4427 , piece_id=RGAV2LRMYPM2ELZ6WD35U7TLKZKYYY5GCKVQX3MHLY3N7QJDVHYA, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304522823 (574h12m19.825346578s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127nGTmG2mLmVxiFuin8vjeHf2TFe9unF75777yZifJiVFXkFX9): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304531891 (574h12m19.825355646s) [noise=true , address=212.102.52.214:4358 , piece_id=NGOQDC5VVPKOVLZJXO3D4GYVMJ4ZHKDYFNY6UKWZSFMOFW74S63Q, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304691776 (574h12m19.825515531s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1Zek7Hwkv4dNwwxz5yKmj9eGCKieCopNGDLQyWHt8wF6dPDNWd): protocol: expected piece hash; context canceled; EOF
May 24 17:00:09.304699961 (574h12m19.825523706s) [noise=true , address=212.102.52.217:4428 , piece_id=454LQ76C4YXS65FCUSEXMRZPGAD5TXVPWGLIX2ZA6UXKC6QFWBYQ, seg_pos={0 0}, upload=550742]: Long tail cancellation due to optimal threshold
May 24 17:00:09.304729618 (574h12m19.825553363s) [seg_pos={0 0}, upload=550742]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 24 17:00:09.304816744 (574h12m19.825640499s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 24 17:00:09.304822455 (574h12m19.82564621s) [upload=550742]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 24 17:00:49.945461696 (574h13m0.466285441s) [upload=550742]: Done uploading segments...
May 24 17:00:49.945480471 (574h13m0.466304216s) [upload=550742]: Upload finished. err: <nil>
Duration: 1m0.002552909s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [X..................XX........................X...........X..] duration:
6.614639118s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:00:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.61466674s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [.X..................XX........................X...........X.] duration:
6.813263621s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 17:00:03 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.813290891s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
5 out of 60 is failed (threshold=3) [..X..................XX........................X...........X] duration:
1m0.000830746s trace:
https://jaeger.dev.storj.io/trace/7ff3407f5d52c65f
FAIL (2026-05-24 16:59:55 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000852508s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...X..................XX........................X...........] duration:
20.635274333s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:58:32 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 20.635338705s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [....X..................XX........................X..........] duration:
27.688517189s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:57:26 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 27.688533008s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.....X..................XX........................X.........] duration:
51.38060183s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:56:58 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 51.380622219s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [......X..................XX........................X........] duration:
6.753899461s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:55:42 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.753911774s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.......X..................XX........................X.......] duration:
6.632293284s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:54:56 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.632310739s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [........X..................XX........................X......] duration:
6.486622712s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:54:02 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.486651048s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.........X..................XX........................X.....] duration:
6.661899567s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:53:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.6619169s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..........X..................XX........................X....] duration:
6.639131192s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:52:35 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.639150949s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [...........X..................XX........................X...] duration:
6.781028524s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:52:17 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.781043162s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [............X..................XX........................X..] duration:
6.757818907s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:51:08 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.757846148s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [.............X..................XX........................X.] duration:
6.588741645s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:50:39 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.58877032s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
4 out of 60 is failed (threshold=3) [..............X..................XX........................X] duration:
1m0.000464625s trace:
https://jaeger.dev.storj.io/trace/59e0a13c37f9bd05
FAIL (2026-05-24 16:50:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000481978s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
3 out of 60 is failed (threshold=3) [...............X..................XX........................] duration:
6.762162511s trace:
https://jaeger.dev.storj.io/trace/282833b08f778c77
FAIL (2026-05-24 16:48:33 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.762186597s
core
uplink
us-select-1
us1-placement-20
|