Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...........................X................X...............] duration:
3.919348339s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:17:12 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 3.919371433s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [.........X....X.......X......X..XX.XX.X....X.....XX......X..] duration:
7.26285184s trace:
https://jaeger.dev.storj.io/trace/698dfada1f99867a
FAIL (2026-05-25 18:17:05 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.262866097s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [................X...........................................] duration:
687.818018ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:16:59 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 687.843816ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [............................X................X..............] duration:
3.859230173s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:16:50 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 3.859281722s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [..........X....X.......X......X..XX.XX.X....X.....XX......X.] duration:
14.972027027s trace:
https://jaeger.dev.storj.io/trace/698dfada1f99867a
FAIL (2026-05-25 18:16:43 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 14.972037577s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [...........X....X.......X......X..XX.XX.X....X.....XX......X] duration:
1m0.000514719s trace:
https://jaeger.dev.storj.io/trace/698dfada1f99867a
FAIL (2026-05-25 18:16:28 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000552321s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.............................X................X.............] duration:
4.028681613s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:16:24 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 4.028704605s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [....................................X.......................] duration:
5.000945577s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:16:15 UTC)
orbiter warning: context deadline exceeded
Duration: 5.000971045s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.................X..........................................] duration:
673.919097ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:15:49 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 673.940659ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..............................X................X............] duration:
4.065583674s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:15:42 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 4.065608259s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.....................................X......................] duration:
3.89516256s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:15:32 UTC)
orbiter warning: context deadline exceeded
Duration: 3.895188681s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..................X.........................................] duration:
675.743639ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:15:22 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 675.762976ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...............................X................X...........] duration:
3.958022577s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:15:08 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 3.958048556s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...................X........................................] duration:
678.413ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:15:02 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 678.438749ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [................................X................X..........] duration:
22.155650823s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:14:59 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 22.155672013s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [....................X.......................................] duration:
681.617417ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:14:46 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 681.628828ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [............X....X.......X......X..XX.XX.X....X.....XX......] duration:
6.908224168s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:14:37 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.908247604s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.....................X......................................] duration:
678.730667ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:14:27 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 678.755616ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [......................................X.....................] duration:
3.748378893s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:14:19 UTC)
orbiter warning: context deadline exceeded
Duration: 3.74840358s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [.............X....X.......X......X..XX.XX.X....X.....XX.....] duration:
6.638687862s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:14:15 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.638711526s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [......................X.....................................] duration:
674.918926ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:14:07 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 674.938724ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.................................X................X.........] duration:
4.301074329s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:13:57 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 4.301098315s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [..................................X................X........] duration:
23.224072073s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:13:39 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 23.224083705s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.......................................X....................] duration:
3.680515514s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:13:30 UTC)
orbiter warning: context deadline exceeded
Duration: 3.680535461s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [..............X....X.......X......X..XX.XX.X....X.....XX....] duration:
6.902019707s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:13:24 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.902045246s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [...................................X................X.......] duration:
41.771053346s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:13:15 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 41.771091329s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.......................X....................................] duration:
682.609108ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:12:54 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 682.631451ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [...............X....X.......X......X..XX.XX.X....X.....XX...] duration:
7.049105586s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:12:30 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 7.049127487s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [........................................X...................] duration:
3.866937333s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:12:24 UTC)
orbiter warning: context deadline exceeded
Duration: 3.866955949s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [.........................................X..................] duration:
3.639961145s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:12:02 UTC)
orbiter warning: context deadline exceeded
Duration: 3.639985302s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [........................X...................................] duration:
681.252098ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:12:00 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 681.280143ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [....................................X................X......] duration:
3.993137975s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:11:43 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 3.993161109s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [.........................X..................................] duration:
672.460534ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:11:42 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 672.513294ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
1 out of 60 is failed (threshold=6) [..........................................X.................] duration:
3.733531403s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:11:35 UTC)
orbiter warning: context deadline exceeded
Duration: 3.73355143s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [................X....X.......X......X..XX.XX.X....X.....XX..] duration:
6.900585362s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:11:27 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.900607474s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.....................................X................X.....] duration:
4.049234049s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:11:11 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 4.049258305s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [X................X....X.......X......X..XX.XX.X....X.....XX.] duration:
6.819879091s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:11:01 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 6.819897036s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
13 out of 60 is failed (threshold=3) [.X................X....X.......X......X..XX.XX.X....X.....XX] duration:
1m0.000713273s trace:
https://jaeger.dev.storj.io/trace/59b9e788be2903db
FAIL (2026-05-25 18:10:54 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.000723132s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [..........................X.................................] duration:
676.014846ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:10:43 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 676.044493ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [......................................X................X....] duration:
37.588803218s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:10:32 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 37.588827415s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [X..........................................X................] duration:
3.875021542s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:10:31 UTC)
orbiter warning: context deadline exceeded
Duration: 3.875479295s
core
uplink
us-select-1
us1-placement-20
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
12 out of 60 is failed (threshold=3) [..X................X....X.......X......X..XX.XX.X....X.....X] duration:
1m0.001459837s trace:
https://jaeger.dev.storj.io/trace/6dcbd02e35d5dcba
FAIL (2026-05-25 18:09:53 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 1m0.001481699s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [...........................X................................] duration:
663.35494ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:09:45 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 663.376341ms
core
satellite
us1
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [.X..........................................X...............] duration:
3.867549589s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:09:41 UTC)
orbiter warning: context deadline exceeded
Duration: 3.867569107s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.......................................X................X...] duration:
3.991422499s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:09:14 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 3.991464379s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails)antiblip:
2 out of 60 is failed (threshold=6) [..X..........................................X..............] duration:
4.777856507s trace:
https://jaeger.dev.storj.io/trace/14eb8131f680ed75
FAIL (2026-05-25 18:08:45 UTC)
orbiter warning: context deadline exceeded
Duration: 4.777878409s
core
uplink
us-select-1
us1-placement-20
|
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)antiblip:
1 out of 60 is failed (threshold=3) [............................X...............................] duration:
661.227247ms target:
https://us1.storj.io/login trace:
https://jaeger.dev.storj.io/trace/7e710747942acdb2
FAIL (2026-05-25 18:08:33 UTC)
orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout
Duration: 661.252336ms
core
satellite
us1
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [........................................X................X..] duration:
7.288394231s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:08:26 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 7.288424019s
core
uplink
uk-select-1
uk1-placement-17
|
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20)antiblip:
11 out of 60 is failed (threshold=3) [...X................X....X.......X......X..XX.XX.X....X.....] duration:
21.421704745s trace:
https://jaeger.dev.storj.io/trace/fac1154c0207b7
FAIL (2026-05-25 18:08:14 UTC)
orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded
Duration: 21.421725815s
core
uplink
us-select-1
us1-placement-20
|
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17)antiblip:
2 out of 60 is failed (threshold=3) [.........................................X................X.] duration:
4.364590385s trace:
https://jaeger.dev.storj.io/trace/467f79c1decb330
FAIL (2026-05-25 18:07:59 UTC)
orbiter warning:
operation: DownloadObject
======ERROR======
uplink: metaclient: context deadline exceeded
======LOGS======
se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece...
May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment
May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments...
May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil>
May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces
May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF
May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold
May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil>
May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_SegmentCommit
May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: *pb.BatchRequestItem_ObjectCommit
May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments...
May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil>
Duration: 4.364601827s
core
uplink
uk-select-1
uk1-placement-17
|