Orbiter History

Orbiter: EU/US/AP/EDGE | Code | Config | Playbook | Escalation Procedure

Orbiter Check History

Showing 50 of 899610 reports (Page 2509 of 17993)

checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [..........X....X.......X......X..XX.XX.X....X.....XX......X.] duration: 14.972027027s trace: https://jaeger.dev.storj.io/trace/698dfada1f99867a FAIL (2026-05-25 18:16:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 14.972037577s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [...........X....X.......X......X..XX.XX.X....X.....XX......X] duration: 1m0.000514719s trace: https://jaeger.dev.storj.io/trace/698dfada1f99867a FAIL (2026-05-25 18:16:28 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000552321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [.............................X................X.............] duration: 4.028681613s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:16:24 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 4.028704605s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [....................................X.......................] duration: 5.000945577s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:16:15 UTC) orbiter warning: context deadline exceeded Duration: 5.000971045s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 673.919097ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:15:49 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 673.940659ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [..............................X................X............] duration: 4.065583674s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:15:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 4.065608259s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.....................................X......................] duration: 3.89516256s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:15:32 UTC) orbiter warning: context deadline exceeded Duration: 3.895188681s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 675.743639ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:15:22 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 675.762976ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [...............................X................X...........] duration: 3.958022577s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:15:08 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 3.958048556s core uplink uk-select-1 uk1-placement-17
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 678.413ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:15:02 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 678.438749ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [................................X................X..........] duration: 22.155650823s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:14:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 22.155672013s core uplink uk-select-1 uk1-placement-17
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 681.617417ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:14:46 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 681.628828ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [............X....X.......X......X..XX.XX.X....X.....XX......] duration: 6.908224168s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:14:37 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.908247604s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 678.730667ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:14:27 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 678.755616ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [......................................X.....................] duration: 3.748378893s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:14:19 UTC) orbiter warning: context deadline exceeded Duration: 3.74840358s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [.............X....X.......X......X..XX.XX.X....X.....XX.....] duration: 6.638687862s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:14:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.638711526s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 674.918926ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:14:07 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 674.938724ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [.................................X................X.........] duration: 4.301074329s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:13:57 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 4.301098315s core uplink uk-select-1 uk1-placement-17
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [..................................X................X........] duration: 23.224072073s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:13:39 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 23.224083705s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.......................................X....................] duration: 3.680515514s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:13:30 UTC) orbiter warning: context deadline exceeded Duration: 3.680535461s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [..............X....X.......X......X..XX.XX.X....X.....XX....] duration: 6.902019707s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:13:24 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.902045246s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [...................................X................X.......] duration: 41.771053346s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:13:15 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 41.771091329s core uplink uk-select-1 uk1-placement-17
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 682.609108ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:12:54 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 682.631451ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [...............X....X.......X......X..XX.XX.X....X.....XX...] duration: 7.049105586s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:12:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 7.049127487s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [........................................X...................] duration: 3.866937333s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:12:24 UTC) orbiter warning: context deadline exceeded Duration: 3.866955949s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [.........................................X..................] duration: 3.639961145s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:12:02 UTC) orbiter warning: context deadline exceeded Duration: 3.639985302s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 681.252098ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:12:00 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 681.280143ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [....................................X................X......] duration: 3.993137975s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:11:43 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 3.993161109s core uplink uk-select-1 uk1-placement-17
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 672.460534ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:11:42 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 672.513294ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 1 out of 60 is failed (threshold=6) [..........................................X.................] duration: 3.733531403s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:11:35 UTC) orbiter warning: context deadline exceeded Duration: 3.73355143s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [................X....X.......X......X..XX.XX.X....X.....XX..] duration: 6.900585362s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:11:27 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.900607474s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X................X.....] duration: 4.049234049s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:11:11 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 4.049258305s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [X................X....X.......X......X..XX.XX.X....X.....XX.] duration: 6.819879091s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:11:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.819897036s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 13 out of 60 is failed (threshold=3) [.X................X....X.......X......X..XX.XX.X....X.....XX] duration: 1m0.000713273s trace: https://jaeger.dev.storj.io/trace/59b9e788be2903db FAIL (2026-05-25 18:10:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.000723132s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 676.014846ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:10:43 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 676.044493ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [......................................X................X....] duration: 37.588803218s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:10:32 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 37.588827415s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [X..........................................X................] duration: 3.875021542s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:10:31 UTC) orbiter warning: context deadline exceeded Duration: 3.875479295s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 12 out of 60 is failed (threshold=3) [..X................X....X.......X......X..XX.XX.X....X.....X] duration: 1m0.001459837s trace: https://jaeger.dev.storj.io/trace/6dcbd02e35d5dcba FAIL (2026-05-25 18:09:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 1m0.001481699s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 663.35494ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:09:45 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 663.376341ms core satellite us1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [.X..........................................X...............] duration: 3.867549589s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:09:41 UTC) orbiter warning: context deadline exceeded Duration: 3.867569107s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X................X...] duration: 3.991422499s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:09:14 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 3.991464379s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [..X..........................................X..............] duration: 4.777856507s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:08:45 UTC) orbiter warning: context deadline exceeded Duration: 4.777878409s core uplink us-select-1 us1-placement-20
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 661.227247ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:08:33 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 661.252336ms core satellite us1
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [........................................X................X..] duration: 7.288394231s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:08:26 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 7.288424019s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 11 out of 60 is failed (threshold=3) [...X................X....X.......X......X..XX.XX.X....X.....] duration: 21.421704745s trace: https://jaeger.dev.storj.io/trace/fac1154c0207b7 FAIL (2026-05-25 18:08:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 21.421725815s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X................X.] duration: 4.364590385s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:07:59 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 4.364601827s core uplink uk-select-1 uk1-placement-17
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/50kb", Size: 50.0 KiB} failure (antiblip window 6/60): (gateway.us1.placement_20_thumbnails) antiblip: 2 out of 60 is failed (threshold=6) [...X..........................................X.............] duration: 4.6325422s trace: https://jaeger.dev.storj.io/trace/14eb8131f680ed75 FAIL (2026-05-25 18:07:57 UTC) orbiter warning: context deadline exceeded Duration: 4.632563671s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12YKhPHptsaNZY8ksCHkfRkP9TBNdeQmjcRdRSZCiZDwoApAShR@uk1.storj.io:7777", Bucket: "orbiter", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.uk1.placement_17) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X................X] duration: 1m0.002467993s trace: https://jaeger.dev.storj.io/trace/467f79c1decb330 FAIL (2026-05-25 18:07:54 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== se=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.813733143 (16h3m2.597073242s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814170728 (16h3m2.597510827s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814532990 (16h3m2.597873089s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.814792664 (16h3m2.598132764s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815026731 (16h3m2.59836683s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815255207 (16h3m2.598595296s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.815526655 (16h3m2.598866754s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817053338 (16h3m2.600393437s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.817184408 (16h3m2.600524507s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818090236 (16h3m2.601430335s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.818269799 (16h3m2.601609898s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819170618 (16h3m2.602510717s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.819559901 (16h3m2.6029s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.820329489 (16h3m2.603669588s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:16.821320470 (16h3m2.604660559s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Uploading piece... May 25 18:07:17.875060315 (16h3m3.658400414s) [upload=15359]: Next returned nil segment May 25 18:07:17.875071437 (16h3m3.658411536s) [upload=15359]: Waiting for error group managing segments... May 25 18:07:18.161378706 (16h3m3.944718805s) [noise=true , address=212.102.52.214:4356 , piece_id=5OQLNMQN2NWE2OJO7FS44O6NI4NRMDCGYQQ3OTTH62CQDZZGHVQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.163883035 (16h3m3.947223134s) [noise=true , address=212.102.52.215:4369 , piece_id=TLNVSD3KWUSXOW3YLEMNRREQW2RMZABFJEEGX64XH7QYAXZBNDHQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169179691 (16h3m3.9525198s) [noise=true , address=212.102.52.214:4347 , piece_id=6SXGEUYVLSOUSGPUY22UAXQKUWFMWVQOV3NHMMOOJY3NNL5XBAHA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.169953397 (16h3m3.953293496s) [noise=true , address=212.102.52.214:4358 , piece_id=NDE6FEYH2OTGRTRIHGJBGJSV77ZUKHXAR4AON3NDKBU33EL4BGZQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.170012670 (16h3m3.953352769s) [noise=true , address=212.102.52.216:4406 , piece_id=DQK2NNDQZWMM7KWBUMYVDJZTGT3SITN3M2ILQ7Y3QNIBV4WZXTTQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173457273 (16h3m3.956797402s) [noise=true , address=212.102.52.217:4431 , piece_id=3JXJF2YLDKTQDXDK666LSTYGYTMHSE3ZXJ7Y5WOUHAXWZVFRBRYA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.173853839 (16h3m3.957193938s) [noise=true , address=212.102.52.213:4340 , piece_id=GGQZRBZNC5DKXOZYV664XOFD2YUAZAL4RPRDQY7ZY6LUN6NKEFKQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.175372487 (16h3m3.958712586s) [noise=true , address=212.102.52.216:4405 , piece_id=CBCNKWI2VLVCUZDVM54E243ULDQUGYEH5EWRK2ES6V6LRUPWQIIA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.181671326 (16h3m3.965011425s) [noise=true , address=212.102.52.215:4376 , piece_id=RK5KEGAOHFZBU4NN7IO44XPBY5YAOANJ5VLPUU3XVBIGVVFSFZVA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.186238400 (16h3m3.969578509s) [noise=true , address=212.102.52.217:4418 , piece_id=EVBLYS5IQFKBL5TE3UTNM5SENH3COOX6W4JDF53SFFON3JOYQHMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.190371015 (16h3m3.973711114s) [noise=true , address=212.102.52.213:4328 , piece_id=GUZPGD4HCE5PSNWBWZMLEDHA5AEMZZTEF7N2V4OXC4LCI7TIRIFQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.191589199 (16h3m3.974929298s) [noise=true , address=212.102.52.216:4408 , piece_id=IEQIPNTTQ3LN7LVVVQSSQU67NLWWG5DC6D5I5W33SKPKPN532DMQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.199054834 (16h3m3.982394933s) [noise=true , address=212.102.52.215:4388 , piece_id=DNXSHYL6IAMUWAUUVCH6WANAI3HSQIMVJYKSSG3YKLKNS4S6DGJQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220275774 (16h3m4.003615873s) [noise=true , address=212.102.52.217:4426 , piece_id=CK4UNDBYQQCEVPTT5WGWHBWFH6CRZO3JO7RZL3RO5NOYDFX3DJNA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.220628726 (16h3m4.003968825s) [noise=true , address=212.102.52.215:4378 , piece_id=6456XMZDE4RFGHOTOUDM2SNQJDOCHHERTJKKQA6R6KU4EQCD74XA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226546558 (16h3m4.009886657s) [noise=true , address=212.102.52.213:4321 , piece_id=QFBCH5MLCAIFPWMHO4NNQJLPIX7QG5LPQJUHG77WMPI6VPIBQIQA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: <nil> May 25 18:07:18.226564292 (16h3m4.009904421s) [seg_pos={0 0}, upload=15359]: Satisfied cohort requirements after 16 pieces May 25 18:07:18.226741169 (16h3m4.010081268s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zavf82Jv8HWUWhuGf2nXnhiF1GDupLBVhu6urZQP6N6j71R3C): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226748514 (16h3m4.010088613s) [noise=true , address=212.102.52.214:4368 , piece_id=OPJWV4TDPYNLLY2A5TSKGCUGX542YUX2OMPMILUTPE3TJR4L72AA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226822314 (16h3m4.010162413s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129SbAN2aTSXzo8JLELV2ssdkmT5k7kbTcvxSZE4sFsVaKw1Aok): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226831161 (16h3m4.01017126s) [noise=true , address=212.102.52.213:4343 , piece_id=JA6JX5GQOL6WT6MQV4G4UZ2M73QAW5ZII5OW5EO367VKYRESOGGA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.226940960 (16h3m4.010281059s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12CPW6Lyzfry6DmU86Fh3rW6u1RS3qpe79mYMaJRB5r4QEEQC6F): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.226946731 (16h3m4.01028683s) [noise=true , address=212.102.52.217:4432 , piece_id=GZKMD3YNEMZNXRNHAPFBBXL6T7XOFUHKVJURDCJQ6CTCZRDUW4KQ, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227019280 (16h3m4.010359379s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:122Phe9CqWEgoJLbxBLeSUDWMMmah4JR5tuyPENLK3GrfKE3xBs): protocol: expected piece hash; context canceled; EOF May 25 18:07:18.227026895 (16h3m4.010366994s) [noise=true , address=212.102.52.216:4416 , piece_id=AMAJVMUOCVS5TOZLHMDOUSQ5YXCFSYEDT7BW4CW56M6VA6LIDJLA, seg_pos={0 0}, upload=15359]: Long tail cancellation due to optimal threshold May 25 18:07:18.227043557 (16h3m4.010383656s) [seg_pos={0 0}, upload=15359]: Done waiting for segment. successful: 16 optimal: 16 errs: <nil> May 25 18:07:18.227113290 (16h3m4.010453389s) [upload=15359]: Flush batch item: pb.BatchRequestItem_SegmentCommit May 25 18:07:18.227118159 (16h3m4.010458258s) [upload=15359]: Flush batch item: pb.BatchRequestItem_ObjectCommit May 25 18:07:34.282635746 (16h3m20.065975866s) [upload=15359]: Done uploading segments... May 25 18:07:34.282653911 (16h3m20.06599402s) [upload=15359]: Upload finished. err: <nil> Duration: 1m0.002484475s core uplink uk-select-1 uk1-placement-17
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 665.659552ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:07:44 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 665.684108ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 661.79303ms target: https://us1.storj.io/login trace: https://jaeger.dev.storj.io/trace/7e710747942acdb2 FAIL (2026-05-25 18:07:22 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 34.4.32.59:443: i/o timeout Duration: 661.809553ms core satellite us1

1
...
2507
2508
2509
2510
2511
...
17993