Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 5.742037678s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:06:32 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.742067203s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 5.70949907s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:05:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.709524238s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 5.554369094s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:04:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.554399672s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 5.699169773s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:04:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.699205282s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 5.641908124s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:03:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.641939875s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 5.644580303s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:02:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.644616694s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 5.676631535s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:01:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.676657984s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.666918852s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:01:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.666949279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 5.78854816s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 07:00:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.788579079s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 5.766293941s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:59:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.766321423s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 5.641121509s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:58:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.641150233s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 5.693527531s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:58:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.69356389s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.634722098s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:57:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.634771782s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 5.666401928s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:56:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.666423779s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 5.504697541s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:55:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.504730533s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.646075365s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:54:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.64611s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.683041575s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:54:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.683073775s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.680624s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:53:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.680651092s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.75023504s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:52:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.750643217s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.668050725s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:51:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.668090941s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.740930658s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:50:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.740952197s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m27.461664045s trace: https://jaeger.dev.storj.io/trace/28b214f115423b77 FAIL (2025-11-30 06:50:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 2s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.402913290 (502h43m43.954383741s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403416118 (502h43m43.954886579s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.403887445 (502h43m43.955357906s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.404543555 (502h43m43.956014016s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405037645 (502h43m43.956508106s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405550622 (502h43m43.957021083s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.405883507 (502h43m43.957353968s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406397115 (502h43m43.957867576s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.406826061 (502h43m43.958296522s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407403771 (502h43m43.958874222s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.407853858 (502h43m43.959324309s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408295048 (502h43m43.959765499s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.408699278 (502h43m43.960169739s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.409105621 (502h43m43.960576083s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.411134606 (502h43m43.962605057s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:45.412039729 (502h43m43.96351019s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Uploading piece... Nov 30 06:47:46.733132764 (502h43m45.284603235s) [upload=442670]: Next returned nil segment Nov 30 06:47:46.733151259 (502h43m45.28462172s) [upload=442670]: Waiting for error group managing segments... Nov 30 06:47:47.072473948 (502h43m45.623944409s) [noise=true , address=64.49.13.154:4935 , piece_id=DD4CGSNRHT6BU5TLJWEKNLJ6EO3FSQRDWCTNVSEKI5QJG2ATXTTQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.282360303 (502h43m45.833830764s) [noise=true , address=145.79.227.14:4844 , piece_id=V3KKSPN3TNJM7PGMEXAGSNVWO3UDD4IAUTKF5B666IJOVYP4YVQA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.301194157 (502h43m45.852664618s) [noise=true , address=77.90.31.11:4583 , piece_id=BK3JAUGZQBOUGPSCTMW7OG6IF6ZUBMIEE5MTFJOWS3LZ6X2QX6QQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.312131063 (502h43m45.863601524s) [noise=true , address=145.79.227.17:4998 , piece_id=2QFE355N6H5CTF467DZ25NKMCMX2URD33SZG4YMUHF2KSIUFCAAQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.322047256 (502h43m45.873517717s) [noise=true , address=77.90.31.10:4539 , piece_id=L523SLJTJFAVPZGRKJTKMS35KFMRFKYIARZAMO7JORTUFZSI3QBQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.331709534 (502h43m45.883179995s) [noise=true , address=77.90.31.16:4745 , piece_id=RY24ZXN2I52GSIOMWMXYKE3J4E63BUF6VUM6OHOAA4RIYDFVRTGA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.334351865 (502h43m45.885822327s) [noise=true , address=77.90.31.17:4793 , piece_id=HSDFWCIZ6B72BGLC4BQZJXLKWWKBXAEFGMJBRVSZHOIXYH6MBPUQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.337367468 (502h43m45.888837929s) [noise=true , address=77.90.31.15:4701 , piece_id=GISXVLZLBW6HWDQ2SQY5UOGZCROUJW5WPXDR2VJKM27N5UYVMHWQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.344891755 (502h43m45.896362227s) [noise=true , address=77.90.31.21:4903 , piece_id=BFEAO6UAACNRWDV5RQNIJYYNF53E25KHZQO57NLHZ6OWWXP6W6YQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.354485234 (502h43m45.905955695s) [noise=true , address=45.91.116.14:4203 , piece_id=WJIWHXMLWNK7Z2M3NQMCKWAZWCSYX7IRLZTAAJ5SQMCESPMG42RA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.363177555 (502h43m45.914648017s) [noise=true , address=77.90.31.18:4821 , piece_id=2HMI3HKYR76PLHQAD7ZWBUDFKXCUOZGAVP7VAOUSZQR2GZ4SCXVA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.395586539 (502h43m45.947057s) [noise=true , address=45.91.116.5:4300 , piece_id=77W6LPIWIWZXYISAIKN2EDHUVBQQXVGCTCL3GV7GF2TUIMUHQYIA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.397780627 (502h43m45.949251088s) [noise=true , address=45.91.116.12:4259 , piece_id=3IYOR4QDCFH6BLFZCGHVTCHMFGCITN7BCV6ACKIR76ZDA67WZMFQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405048085 (502h43m45.956518577s) [noise=true , address=45.91.116.13:4186 , piece_id=BU6RHXLFDGVRHRCWXTYKK52SGJTSCTWWXLHIVZWLZYFJUY3TON5Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: <nil> Nov 30 06:47:47.405079676 (502h43m45.956550127s) [seg_pos={0 0}, upload=442670]: Segment reached optimal threshold of 14 pieces. Nov 30 06:47:47.405376721 (502h43m45.956847182s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405387472 (502h43m45.956857933s) [noise=true , address=45.91.116.7:4035 , piece_id=Z5HOMMHMDWZBUOKVIAJKW3NVMXKASC5A3AGAJFO46DUFT5NFHPKQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405447586 (502h43m45.956918037s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qJRodezwvD9yoBUTTiWsqqAbW3YCcZ7DBJEXCqPebkeRkb3QA): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405468997 (502h43m45.956939458s) [noise=true , address=77.90.31.13:4643 , piece_id=FS3QNTULLAZBNIDP5Z3LFYZRE2PBEQLBBI4OI53OORPL2K5BHELQ, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405658237 (502h43m45.957128688s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12VQKskrBadKVwYpngNNeSJPn3knJs7WXtfz9D6syskCDz9EVSt): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405664610 (502h43m45.957135061s) [noise=true , address=45.91.116.6:4984 , piece_id=NOPGTIS4E2UCGRI7L2IG2MN4SP3QT3DMAGZT4C6NVNX5VBRGST2Q, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405888796 (502h43m45.957359257s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1zHWknNPWJ79pDmU1vfv8TCqWKU9sxrP4upTRBNsPnc8LcLgzK): protocol: expected piece hash; context canceled; EOF Nov 30 06:47:47.405898344 (502h43m45.957368795s) [noise=true , address=64.49.13.169:4715 , piece_id=AXLC2DPJZ2ECUK6FTH7Z4UDMBPDGLVUU5IFKPQHGBEK5BLHUJ3QA, seg_pos={0 0}, upload=442670]: Long tail cancellation due to optimal threshold Nov 30 06:47:47.405920366 (502h43m45.957390827s) [seg_pos={0 0}, upload=442670]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Nov 30 06:47:47.406003544 (502h43m45.957473995s) [upload=442670]: Flush batch item: pb.BatchRequestItem_SegmentCommit Nov 30 06:47:47.406009336 (502h43m45.957479797s) [upload=442670]: Flush batch item: pb.BatchRequestItem_ObjectCommit Nov 30 06:50:11.908963016 (502h46m10.460433487s) [upload=442670]: Done uploading segments... Nov 30 06:50:11.909048348 (502h46m10.460518819s) [upload=442670]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m27.461686599s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.994970878s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:57:20 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.995004062s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.988518423s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:56:34 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.988630416s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.863983588s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:56:05 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.864017743s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.860665697s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:54:53 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.860691576s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.002854826s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:53:43 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 6.00289335s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.039324026s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:53:17 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 6.039358652s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.904087172s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:52:24 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.904117158s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.906261589s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:51:15 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.906296836s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.687610679s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:50:45 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.687643251s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.053084493s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:50:24 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 6.053118799s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.212236523s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:50:05 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 6.21226632s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 5.562059518s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:49:43 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.56208668s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 5.701213342s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:48:51 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.701246004s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 5.782796049s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:48:33 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.782878957s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 5.748945904s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:48:16 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.748971492s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 5.786039267s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:47:16 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.786068062s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 5.753162541s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:46:18 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.753189513s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 5.880475168s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:45:47 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.880500966s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 5.673468731s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:44:51 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.673503057s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 5.565273178s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:43:40 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.565302464s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 5.571986863s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:42:58 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.57202686s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 5.818638667s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:42:08 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.818674226s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 5.828905642s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:41:09 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.828939356s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 5.823047405s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:40:44 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.823077702s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 5.668158209s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:39:55 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.668182506s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 5.943317179s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:39:32 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.943345092s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 5.898702063s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:38:57 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.898729755s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 5.824008746s trace: https://jaeger.dev.storj.io/trace/77849155ced6ab16 FAIL (2025-11-30 03:38:28 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 5.824031218s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)