Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 29 out of 60 is failed (threshold=3) [....XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X.XXXXX] duration: 30.001633919s FAIL (2025-09-06 14:26:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:25:36.358029204 (57m12.397602324s) [upload=824]: Starting upload Sep 6 14:25:36.358090100 (57m12.39766322s) [upload=824]: Done starting upload Sep 6 14:25:36.358186324 (57m12.397759444s) [upload=824]: Uploading segments... Sep 6 14:25:36.358271878 (57m12.397844998s) [upload=824]: Got next segment. Inline: false Sep 6 14:25:36.358297187 (57m12.397870337s) [upload=824]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:25:36.358302036 (57m12.397875156s) [upload=824]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:25:39.646878213 (57m15.686451333s) [seg_pos={0 0}, upload=824]: Begin upload segment... Sep 6 14:25:39.646924171 (57m15.686497291s) [seg_pos={0 0}, upload=824]: Done begin upload segment. Sep 6 14:25:39.646937718 (57m15.686510838s) [noise=true , address=45.91.116.14:4189 , piece_id=RBGZGX2J32JQDZTI3RRKRBELPI2HLZ7TS2NRHE7EWMMALH5LDJ7Q, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.646945682 (57m15.686518803s) [noise=true , address=45.91.116.5:4299 , piece_id=TZ7GOJ3VCQOHSP3MPVMVUBVE2MXQEHONDEWCRJYDBCTXTS3E6IXQ, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.647301083 (57m15.686874203s) [noise=true , address=77.90.31.18:4807 , piece_id=BSSYP7A5JLQDCBAO6E57EV5VMRP6WEU6I6HYIBHOQ37ZL34TKXAQ, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.647308857 (57m15.686881977s) [noise=true , address=77.90.31.16:4735 , piece_id=U6HVJB2Z5MX3KZ7N2O3AJOMGDC2R2O3XLAMFBJALLGIGONO4ZFMA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.647678405 (57m15.687251605s) [noise=true , address=45.91.116.14:4191 , piece_id=UC445OSQTA6M3FUZCGB5YCGSRILLSRAWEBCIITBQXQ2OBRFNH5RQ, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.648003397 (57m15.687576507s) [noise=true , address=45.91.116.13:4154 , piece_id=VWHEQKLN7R3P4H6EUDRHGUUELUSDNTWCRNHWYUIFZF3SF7IJGJRA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.648319681 (57m15.687892791s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7W6RBAMTNLQESZU2XYTDPH2TDCKJIGEIHTQWBM64MXT3LCXQGA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.648546124 (57m15.688119244s) [noise=true , address=45.91.116.14:4190 , piece_id=SG3XDV4F7VYTAHJH26UCOM6ICPX6ZLDLSWBJYJI247AKYZI6V5UA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.648759964 (57m15.688333074s) [noise=true , address=45.91.116.9:4263 , piece_id=6PJWWBUOJ5OUR3GEGYYMKQJSWWIRULTBA4A33JIRIRNELDWMNWHA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.648976679 (57m15.688549799s) [noise=true , address=45.91.116.11:4119 , piece_id=YEMRKIMZMTEO3PSMVOD3TZXZFXCHNFXBLBSKM3JEXGCICKZV5BDA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.649196289 (57m15.688769409s) [noise=true , address=77.90.31.10:4519 , piece_id=EKCNHIZV3DIOMAXS45IS3DG62CX7OWIHEGLNAEINUW3YG2AUCHZQ, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.649405399 (57m15.688978509s) [noise=true , address=45.91.116.7:4010 , piece_id=HTYVF7UXLIRJD766KWXDJZACJ66PUWEQQ7CA6ABUAFUTVXQOMCGA, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.649590433 (57m15.689163554s) [noise=true , address=45.91.116.4:4939 , piece_id=NR7RYE362TJQGZUTNZQXSBSO3TTGHHZCGV4RLDJWAIDUYTCJH64Q, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.649847084 (57m15.689420204s) [noise=true , address=77.90.31.15:4699 , piece_id=HXPIDMJ5QQSL25SVS7M5TYEI3OLXO4VNNM6CRA547VWRCBL7FWNQ, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.650047948 (57m15.689621068s) [noise=true , address=45.91.116.12:4226 , piece_id=QLECCUWKP43RMZ3LBGLV5DCMMCM5SIFZSXWHVG3QZKCSV2DIKB4A, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:39.650234385 (57m15.689807505s) [noise=true , address=45.91.116.8:4048 , piece_id=JR2V5VELNH3NITXC535Y5QMU42YU7SCS7XURIE4LR5LPOWC3OOAQ, seg_pos={0 0}, upload=824]: Uploading piece... Sep 6 14:25:41.530393920 (57m17.56996705s) [upload=824]: Next returned nil segment Sep 6 14:25:41.530415301 (57m17.569988421s) [upload=824]: Waiting for error group managing segments... Sep 6 14:25:41.828924085 (57m17.868497205s) [noise=true , address=45.91.116.14:4189 , piece_id=RBGZGX2J32JQDZTI3RRKRBELPI2HLZ7TS2NRHE7EWMMALH5LDJ7Q, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.853951957 (57m17.893525077s) [noise=true , address=45.91.116.8:4048 , piece_id=JR2V5VELNH3NITXC535Y5QMU42YU7SCS7XURIE4LR5LPOWC3OOAQ, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.854916572 (57m17.894489692s) [noise=true , address=45.91.116.11:4119 , piece_id=YEMRKIMZMTEO3PSMVOD3TZXZFXCHNFXBLBSKM3JEXGCICKZV5BDA, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.866703498 (57m17.906276608s) [noise=true , address=77.90.31.18:4807 , piece_id=BSSYP7A5JLQDCBAO6E57EV5VMRP6WEU6I6HYIBHOQ37ZL34TKXAQ, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.874462445 (57m17.914035565s) [noise=true , address=45.91.116.5:4299 , piece_id=TZ7GOJ3VCQOHSP3MPVMVUBVE2MXQEHONDEWCRJYDBCTXTS3E6IXQ, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.905130248 (57m17.944703418s) [noise=true , address=45.91.116.9:4263 , piece_id=6PJWWBUOJ5OUR3GEGYYMKQJSWWIRULTBA4A33JIRIRNELDWMNWHA, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.925508232 (57m17.965081352s) [noise=true , address=45.91.116.12:4226 , piece_id=QLECCUWKP43RMZ3LBGLV5DCMMCM5SIFZSXWHVG3QZKCSV2DIKB4A, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.928963684 (57m17.968536804s) [noise=true , address=77.90.31.15:4699 , piece_id=HXPIDMJ5QQSL25SVS7M5TYEI3OLXO4VNNM6CRA547VWRCBL7FWNQ, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:41.982032171 (57m18.021605292s) [noise=true , address=45.91.116.12:4227 , piece_id=4H7W6RBAMTNLQESZU2XYTDPH2TDCKJIGEIHTQWBM64MXT3LCXQGA, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:42.016889402 (57m18.056462522s) [noise=true , address=45.91.116.4:4939 , piece_id=NR7RYE362TJQGZUTNZQXSBSO3TTGHHZCGV4RLDJWAIDUYTCJH64Q, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:42.105006877 (57m18.144579998s) [noise=true , address=77.90.31.10:4519 , piece_id=EKCNHIZV3DIOMAXS45IS3DG62CX7OWIHEGLNAEINUW3YG2AUCHZQ, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:42.113259800 (57m18.15283292s) [noise=true , address=45.91.116.7:4010 , piece_id=HTYVF7UXLIRJD766KWXDJZACJ66PUWEQQ7CA6ABUAFUTVXQOMCGA, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:42.184151372 (57m18.223724492s) [noise=true , address=45.91.116.14:4190 , piece_id=SG3XDV4F7VYTAHJH26UCOM6ICPX6ZLDLSWBJYJI247AKYZI6V5UA, seg_pos={0 0}, upload=824]: Done uploading piece. err: <nil> Sep 6 14:25:42.184172061 (57m18.223745181s) [seg_pos={0 0}, upload=824]: Segment reached optimal threshold of 13 pieces. Sep 6 14:25:42.184513645 (57m18.224086785s) [noise=true , address=77.90.31.16:4735 , piece_id=U6HVJB2Z5MX3KZ7N2O3AJOMGDC2R2O3XLAMFBJALLGIGONO4ZFMA, seg_pos={0 0}, upload=824]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h): protocol: read tcp 188.166.225.170:43640->77.90.31.16:4735: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 14:25:42.184628635 (57m18.224201745s) [noise=true , address=45.91.116.14:4191 , piece_id=UC445OSQTA6M3FUZCGB5YCGSRILLSRAWEBCIITBQXQ2OBRFNH5RQ, seg_pos={0 0}, upload=824]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:25:42.184732193 (57m18.224305313s) [noise=true , address=45.91.116.13:4154 , piece_id=VWHEQKLN7R3P4H6EUDRHGUUELUSDNTWCRNHWYUIFZF3SF7IJGJRA, seg_pos={0 0}, upload=824]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: write tcp 188.166.225.170:41248->45.91.116.13:4154: use of closed network connection; write tcp 188.166.225.170:41248->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41248->45.91.116.13:4154: use of closed network connection Sep 6 14:25:42.184748174 (57m18.224321324s) [seg_pos={0 0}, upload=824]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:25:42.184884374 (57m18.224457534s) [upload=824]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:25:42.184891639 (57m18.224464769s) [upload=824]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:25:43.710515049 (57m19.750088209s) [upload=824]: Done uploading segments... Sep 6 14:25:43.710542732 (57m19.750115892s) [upload=824]: Upload finished. err: <nil> Duration: 30.00166042s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 28 out of 60 is failed (threshold=3) [.....XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X.XXXX] duration: 30.001353972s FAIL (2025-09-06 14:25:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:24:53.292193420 (56m29.33176655s) [upload=814]: Starting upload Sep 6 14:24:53.292250138 (56m29.331823268s) [upload=814]: Done starting upload Sep 6 14:24:53.292339320 (56m29.33191246s) [upload=814]: Uploading segments... Sep 6 14:24:53.292436034 (56m29.332009154s) [upload=814]: Got next segment. Inline: false Sep 6 14:24:53.292464038 (56m29.332037158s) [upload=814]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:24:53.292641869 (56m29.332214999s) [upload=814]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:24:53.843422031 (56m29.882995151s) [seg_pos={0 0}, upload=814]: Begin upload segment... Sep 6 14:24:53.843459763 (56m29.883032883s) [seg_pos={0 0}, upload=814]: Done begin upload segment. Sep 6 14:24:53.843495300 (56m29.88306841s) [noise=true , address=45.91.116.7:4011 , piece_id=2O74H6265V32OO4G4FBSSRWX3FCXBJHKYAEOSGCHOA656MQUNFDA, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.843504609 (56m29.883077729s) [noise=true , address=45.91.116.6:4973 , piece_id=3G6SCK5MOG7TWECHQSIJM6UDWEVVYF6LZL3KZJ7MBDDEOB74HVGQ, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.843611653 (56m29.883184773s) [noise=true , address=77.90.31.16:4735 , piece_id=L3HKEMVRYISOVZS33MQZZUXEJHBX7TUOJA727HYFJDHUYCIGAZ7Q, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.843865700 (56m29.88343882s) [noise=true , address=45.91.116.8:4048 , piece_id=ESX566ADHYZKMKRKQZBP2ATRPW26FZXT7ADHQBAW3EVMA5A23O3Q, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.843918020 (56m29.88349114s) [noise=true , address=45.91.116.12:4225 , piece_id=P2P2XBAA7HYFYNLF7REJX23M5UYPHUKWMWZNYTAIDYJ74SUXZVOQ, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844104928 (56m29.883678038s) [noise=true , address=77.90.31.18:4807 , piece_id=4RIVSYLNQTTVPGUPGR57NRM4KMPUBEVZODZK644CBUFYI2TFRZHQ, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844175161 (56m29.883748282s) [noise=true , address=45.91.116.4:4938 , piece_id=FJGWFDKJFERMU2YAS34MOSTPEQGJOZXZOOTJFVAT3LZP3AZUDVWA, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844322323 (56m29.883895443s) [noise=true , address=45.91.116.6:4974 , piece_id=Y56X6XBQHD27OFD46PNVXFI3Z6B4TI3SD2LWWVNJSOT5GO3IUBHA, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844393219 (56m29.883966339s) [noise=true , address=45.91.116.10:4082 , piece_id=KSX2SJA4SJODJ47WXQ5NMTHFZ5XU2HXJAC6YDBC7XX3FLFSMU67A, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844578333 (56m29.884151453s) [noise=true , address=45.91.116.10:4081 , piece_id=INRIZILL6J6VAKRQXEWVCZU5QSDVLNLJKHXCFCUG3WARTY5R3VUQ, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844684617 (56m29.884257737s) [noise=true , address=45.91.116.4:4939 , piece_id=ZYQYK5ZLZR5L5SGXS3A7TTTOVZL3AJGMW2DEKTCH4BOEPCGAJD5Q, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.844820497 (56m29.884393617s) [noise=true , address=45.91.116.14:4191 , piece_id=7WQINSTXQCCZ4CU2UW2EQEOVNEVZZB5NROU7S2U5JQVF4AH4QGAA, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.845046028 (56m29.884619148s) [noise=true , address=77.90.31.19:4841 , piece_id=LH6HHXDSN7VNPZX3BNVDDTYHXT5DRBECMI6SH5VTIH7HGGMM3GDQ, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.845272441 (56m29.884845561s) [noise=true , address=45.91.116.12:4226 , piece_id=OT75TYIS7WH7GTRA5UPJYQPE2COARMKVTVNVA6R6R6FQINFT7HSQ, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.845476732 (56m29.885049852s) [noise=true , address=45.91.116.13:4154 , piece_id=YTPSTHA53RNDXIXKGEZG7JINEC3FY6EBBPRZ33QZD2SSTLQOLBQA, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:53.845677486 (56m29.885250596s) [noise=true , address=45.91.116.11:4119 , piece_id=ULNLRGNPUT3IJBW4B7WUCPBKW3UPLI3RI4RFACNSY54CZTO6ZPZA, seg_pos={0 0}, upload=814]: Uploading piece... Sep 6 14:24:55.721644479 (56m31.761217599s) [upload=814]: Next returned nil segment Sep 6 14:24:55.721664958 (56m31.761238078s) [upload=814]: Waiting for error group managing segments... Sep 6 14:24:56.021719921 (56m32.061293042s) [noise=true , address=45.91.116.14:4191 , piece_id=7WQINSTXQCCZ4CU2UW2EQEOVNEVZZB5NROU7S2U5JQVF4AH4QGAA, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.054706382 (56m32.094279502s) [noise=true , address=45.91.116.12:4225 , piece_id=P2P2XBAA7HYFYNLF7REJX23M5UYPHUKWMWZNYTAIDYJ74SUXZVOQ, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.074353477 (56m32.113926607s) [noise=true , address=45.91.116.4:4939 , piece_id=ZYQYK5ZLZR5L5SGXS3A7TTTOVZL3AJGMW2DEKTCH4BOEPCGAJD5Q, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.086298344 (56m32.125871464s) [noise=true , address=45.91.116.10:4081 , piece_id=INRIZILL6J6VAKRQXEWVCZU5QSDVLNLJKHXCFCUG3WARTY5R3VUQ, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.094177181 (56m32.133750301s) [noise=true , address=45.91.116.11:4119 , piece_id=ULNLRGNPUT3IJBW4B7WUCPBKW3UPLI3RI4RFACNSY54CZTO6ZPZA, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.096901363 (56m32.136474473s) [noise=true , address=45.91.116.6:4974 , piece_id=Y56X6XBQHD27OFD46PNVXFI3Z6B4TI3SD2LWWVNJSOT5GO3IUBHA, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.105736109 (56m32.145309229s) [noise=true , address=45.91.116.13:4154 , piece_id=YTPSTHA53RNDXIXKGEZG7JINEC3FY6EBBPRZ33QZD2SSTLQOLBQA, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.206145334 (56m32.245718454s) [noise=true , address=77.90.31.16:4735 , piece_id=L3HKEMVRYISOVZS33MQZZUXEJHBX7TUOJA727HYFJDHUYCIGAZ7Q, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.220645781 (56m32.260218901s) [noise=true , address=45.91.116.12:4226 , piece_id=OT75TYIS7WH7GTRA5UPJYQPE2COARMKVTVNVA6R6R6FQINFT7HSQ, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.283498651 (56m32.323071771s) [noise=true , address=77.90.31.19:4841 , piece_id=LH6HHXDSN7VNPZX3BNVDDTYHXT5DRBECMI6SH5VTIH7HGGMM3GDQ, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.386046439 (56m32.425619599s) [noise=true , address=45.91.116.10:4082 , piece_id=KSX2SJA4SJODJ47WXQ5NMTHFZ5XU2HXJAC6YDBC7XX3FLFSMU67A, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.396028711 (56m32.435601831s) [noise=true , address=45.91.116.7:4011 , piece_id=2O74H6265V32OO4G4FBSSRWX3FCXBJHKYAEOSGCHOA656MQUNFDA, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.712904601 (56m32.752477721s) [noise=true , address=77.90.31.18:4807 , piece_id=4RIVSYLNQTTVPGUPGR57NRM4KMPUBEVZODZK644CBUFYI2TFRZHQ, seg_pos={0 0}, upload=814]: Done uploading piece. err: <nil> Sep 6 14:24:56.712931523 (56m32.752504643s) [seg_pos={0 0}, upload=814]: Segment reached optimal threshold of 13 pieces. Sep 6 14:24:56.713108892 (56m32.752682012s) [noise=true , address=45.91.116.6:4973 , piece_id=3G6SCK5MOG7TWECHQSIJM6UDWEVVYF6LZL3KZJ7MBDDEOB74HVGQ, seg_pos={0 0}, upload=814]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV): protocol: expected piece hash; context canceled; EOF Sep 6 14:24:56.713139932 (56m32.752713052s) [noise=true , address=45.91.116.4:4938 , piece_id=FJGWFDKJFERMU2YAS34MOSTPEQGJOZXZOOTJFVAT3LZP3AZUDVWA, seg_pos={0 0}, upload=814]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Sep 6 14:24:56.713175961 (56m32.752749071s) [noise=true , address=45.91.116.8:4048 , piece_id=ESX566ADHYZKMKRKQZBP2ATRPW26FZXT7ADHQBAW3EVMA5A23O3Q, seg_pos={0 0}, upload=814]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 6 14:24:56.713208753 (56m32.752781863s) [seg_pos={0 0}, upload=814]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:24:56.713312833 (56m32.752885953s) [upload=814]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:24:56.713319385 (56m32.752892505s) [upload=814]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:24:57.125108715 (56m33.164681835s) [upload=814]: Done uploading segments... Sep 6 14:24:57.125126930 (56m33.16470005s) [upload=814]: Upload finished. err: <nil> Duration: 30.001418375s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 27 out of 60 is failed (threshold=3) [......XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X.XXX] duration: 30.002078029s FAIL (2025-09-06 14:24:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:23:59.266268236 (55m35.305841356s) [upload=801]: Starting upload Sep 6 14:23:59.266349551 (55m35.305922671s) [upload=801]: Done starting upload Sep 6 14:23:59.266484670 (55m35.30605778s) [upload=801]: Uploading segments... Sep 6 14:23:59.266675495 (55m35.306248615s) [upload=801]: Got next segment. Inline: false Sep 6 14:23:59.266735210 (55m35.30630833s) [upload=801]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:23:59.266741061 (55m35.306314181s) [upload=801]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:23:59.581831946 (55m35.621405066s) [seg_pos={0 0}, upload=801]: Begin upload segment... Sep 6 14:23:59.581875930 (55m35.62144906s) [seg_pos={0 0}, upload=801]: Done begin upload segment. Sep 6 14:23:59.581900667 (55m35.621473787s) [noise=true , address=77.90.31.21:4878 , piece_id=I5CNEFDC7X326ZZYI3DTE6HZ5KNBUQJKLNMBNZBXNILESFAGXXJA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.581953679 (55m35.621526799s) [noise=true , address=77.90.31.13:4626 , piece_id=ZHEFWZW5CTJVEZBHIHKV3PRBXUJIHI7W3U7XA3ZEI7SH4B2LI4LA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.582291896 (55m35.621865016s) [noise=true , address=77.90.31.12:4589 , piece_id=BUQRXKLLTJM36NN33SNKOBPXIM4DAPZXKVTVSWNOQSKF575QA4QA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.582440130 (55m35.62201325s) [noise=true , address=45.91.116.13:4154 , piece_id=5BPKT5V7UYSVS6C4YCVMSUVCP3OOWSA5JVU5TSKX43CVLOR6QSEA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.582572233 (55m35.622145353s) [noise=true , address=45.91.116.5:4297 , piece_id=4WADOWCPJOGVIO7SY5PHRFC4UCYW2RTFMLJGJ2UDBR6NP5LWDXMA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.582735605 (55m35.622308725s) [noise=true , address=45.91.116.12:4227 , piece_id=V5AGNSCCZ6SUFIL2RZGH7ZXL25ZKHCARTZQWV6V2QIF2KEJV3MLA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.582956247 (55m35.622529367s) [noise=true , address=45.91.116.14:4190 , piece_id=WAXMKKQP2EJSPD5ATV2ZBBWBYZS5E6CC6DS5WVXWYYOIQQ5V556A, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583011914 (55m35.622585024s) [noise=true , address=45.91.116.14:4191 , piece_id=247KPOEN77HVZ3QLQQKO7ROZ555JF7VXGJOIOORSGRBRXMEQQS6A, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583213780 (55m35.6227869s) [noise=true , address=45.91.116.10:4081 , piece_id=547BTOQ443OZEFA2HZVATGXCWO33QBQS2JNTHMYVGGZ2HXVRPV2A, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583294965 (55m35.622868085s) [noise=true , address=45.91.116.7:4010 , piece_id=EYPHRAULTSL3GVKDO74LZQRYMZSPHUEMXBXT3GDPI7WNEVNZQQKA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583489768 (55m35.623062888s) [noise=true , address=45.91.116.4:4939 , piece_id=KPGJW3V2SRJDZWBO6YKHTTWMJLTYLJKD5GBLL5MHH6QPBRZPI4BA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583609427 (55m35.623182547s) [noise=true , address=45.91.116.13:4153 , piece_id=G3SCFU5ZVWUCDISJEFR27F6O4TF5E6XNA5O73ENHTODXHXEX6GHA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583814088 (55m35.623387208s) [noise=true , address=45.91.116.9:4263 , piece_id=TL3SI24WYRWGKPJC3XRQOZBEEHNDGVSHB4K7HX35HEZCEWLHVMRA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.583889483 (55m35.623462603s) [noise=true , address=45.91.116.6:4973 , piece_id=OGD3OT6NQOLOJQQWVU5OPKNZ6OXGALVCTWYFPW4XDTKUXTDPLFZA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.584061041 (55m35.623634161s) [noise=true , address=45.91.116.8:4045 , piece_id=22GITD354S6CFKQHPPKVJRVFVKAQ5IBWRVPSAGGUILIKA56OPB2Q, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:23:59.584276944 (55m35.623850064s) [noise=true , address=77.90.31.10:4518 , piece_id=BMTLJBCF5I2SAGGCU5IDDVKKSZ4JITH73QXM35GA6XDKIOTZZVFA, seg_pos={0 0}, upload=801]: Uploading piece... Sep 6 14:24:01.436180181 (55m37.475753301s) [upload=801]: Next returned nil segment Sep 6 14:24:01.436196321 (55m37.475769441s) [upload=801]: Waiting for error group managing segments... Sep 6 14:24:01.770785527 (55m37.810358647s) [noise=true , address=45.91.116.4:4939 , piece_id=KPGJW3V2SRJDZWBO6YKHTTWMJLTYLJKD5GBLL5MHH6QPBRZPI4BA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.791265306 (55m37.830838436s) [noise=true , address=45.91.116.13:4154 , piece_id=5BPKT5V7UYSVS6C4YCVMSUVCP3OOWSA5JVU5TSKX43CVLOR6QSEA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.791860835 (55m37.831433955s) [noise=true , address=45.91.116.5:4297 , piece_id=4WADOWCPJOGVIO7SY5PHRFC4UCYW2RTFMLJGJ2UDBR6NP5LWDXMA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.804010113 (55m37.843583243s) [noise=true , address=77.90.31.12:4589 , piece_id=BUQRXKLLTJM36NN33SNKOBPXIM4DAPZXKVTVSWNOQSKF575QA4QA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.847180922 (55m37.886754042s) [noise=true , address=45.91.116.9:4263 , piece_id=TL3SI24WYRWGKPJC3XRQOZBEEHNDGVSHB4K7HX35HEZCEWLHVMRA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.851997879 (55m37.891570999s) [noise=true , address=77.90.31.10:4518 , piece_id=BMTLJBCF5I2SAGGCU5IDDVKKSZ4JITH73QXM35GA6XDKIOTZZVFA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.855628887 (55m37.895202007s) [noise=true , address=45.91.116.12:4227 , piece_id=V5AGNSCCZ6SUFIL2RZGH7ZXL25ZKHCARTZQWV6V2QIF2KEJV3MLA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.856623680 (55m37.8961968s) [noise=true , address=45.91.116.10:4081 , piece_id=547BTOQ443OZEFA2HZVATGXCWO33QBQS2JNTHMYVGGZ2HXVRPV2A, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.864340347 (55m37.903913467s) [noise=true , address=45.91.116.7:4010 , piece_id=EYPHRAULTSL3GVKDO74LZQRYMZSPHUEMXBXT3GDPI7WNEVNZQQKA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.866236955 (55m37.905810085s) [noise=true , address=45.91.116.6:4973 , piece_id=OGD3OT6NQOLOJQQWVU5OPKNZ6OXGALVCTWYFPW4XDTKUXTDPLFZA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.874882439 (55m37.914455569s) [noise=true , address=45.91.116.14:4191 , piece_id=247KPOEN77HVZ3QLQQKO7ROZ555JF7VXGJOIOORSGRBRXMEQQS6A, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.881474724 (55m37.921047844s) [noise=true , address=45.91.116.13:4153 , piece_id=G3SCFU5ZVWUCDISJEFR27F6O4TF5E6XNA5O73ENHTODXHXEX6GHA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.928337175 (55m37.967910295s) [noise=true , address=77.90.31.13:4626 , piece_id=ZHEFWZW5CTJVEZBHIHKV3PRBXUJIHI7W3U7XA3ZEI7SH4B2LI4LA, seg_pos={0 0}, upload=801]: Done uploading piece. err: <nil> Sep 6 14:24:01.928373365 (55m37.967946495s) [seg_pos={0 0}, upload=801]: Segment reached optimal threshold of 13 pieces. Sep 6 14:24:01.928572736 (55m37.968145866s) [noise=true , address=45.91.116.8:4045 , piece_id=22GITD354S6CFKQHPPKVJRVFVKAQ5IBWRVPSAGGUILIKA56OPB2Q, seg_pos={0 0}, upload=801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: expected piece hash; context canceled; EOF Sep 6 14:24:01.928677247 (55m37.968250367s) [noise=true , address=45.91.116.14:4190 , piece_id=WAXMKKQP2EJSPD5ATV2ZBBWBYZS5E6CC6DS5WVXWYYOIQQ5V556A, seg_pos={0 0}, upload=801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: write tcp 188.166.225.170:57894->45.91.116.14:4190: use of closed network connection; write tcp 188.166.225.170:57894->45.91.116.14:4190: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57894->45.91.116.14:4190: use of closed network connection Sep 6 14:24:01.928692535 (55m37.968265665s) [noise=true , address=77.90.31.21:4878 , piece_id=I5CNEFDC7X326ZZYI3DTE6HZ5KNBUQJKLNMBNZBXNILESFAGXXJA, seg_pos={0 0}, upload=801]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 14:24:01.928710249 (55m37.96828337s) [seg_pos={0 0}, upload=801]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:24:01.928830028 (55m37.968403148s) [upload=801]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:24:01.928836311 (55m37.968409441s) [upload=801]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:24:02.286321147 (55m38.325894267s) [upload=801]: Done uploading segments... Sep 6 14:24:02.286344612 (55m38.325917733s) [upload=801]: Upload finished. err: <nil> Duration: 30.002102406s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [.......XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X.XX] duration: 30.002665783s FAIL (2025-09-06 14:23:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:23:27.264140211 (55m3.303713321s) [upload=795]: Starting upload Sep 6 14:23:27.264208762 (55m3.303781872s) [upload=795]: Done starting upload Sep 6 14:23:27.264344321 (55m3.303917441s) [upload=795]: Uploading segments... Sep 6 14:23:27.265728570 (55m3.30530169s) [upload=795]: Got next segment. Inline: false Sep 6 14:23:27.265771251 (55m3.305344371s) [upload=795]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:23:27.265776322 (55m3.305349442s) [upload=795]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:23:27.509637176 (55m3.549210316s) [seg_pos={0 0}, upload=795]: Begin upload segment... Sep 6 14:23:27.509692812 (55m3.549265932s) [seg_pos={0 0}, upload=795]: Done begin upload segment. Sep 6 14:23:27.509716899 (55m3.549290019s) [noise=true , address=45.91.116.13:4154 , piece_id=YIX5O3FX7G2FN5TYV62DSA2VILKWB5WE7LMVQSHK6HNLPYSEH2GA, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.509842558 (55m3.549415688s) [noise=true , address=45.91.116.9:4261 , piece_id=YLM3DM3MICEIRR2YK6ERCWAYC4UIVF53DIR2XJ4ZEWMTHEG3WSZQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.510147763 (55m3.549720893s) [noise=true , address=45.91.116.5:4297 , piece_id=ZY6QNRWYG2XN7J6RWLBDJDIQHU7QFIZFRMHKUJQ6ENHLKII352BQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.510415575 (55m3.549988695s) [noise=true , address=45.91.116.9:4262 , piece_id=QRIEQRJZ23HJIOUHRTZ277G7U6JSJZ4E4AQLFV67YP3PI2BLPFNQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.510501098 (55m3.550074218s) [noise=true , address=77.90.31.14:4663 , piece_id=AT5XH7MX6UKIVAFAZD2BPMF55JFDVSQRSRCELU7W5AI6W7YS6NLQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.510679200 (55m3.55025232s) [noise=true , address=77.90.31.12:4592 , piece_id=ERYQTRSCQYS7I44QDFYZQZ6OIT5IIE2ESZHVYHEPQBWUZEXXZECA, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.510880545 (55m3.550453665s) [noise=true , address=45.91.116.10:4083 , piece_id=QZHEMQ7EIQ6HV72IOSYWAM3QB44KS3VSBSMRWF7GYZLACWW4ZU3A, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.510915863 (55m3.550488983s) [noise=true , address=77.90.31.12:4590 , piece_id=4WZHDYEAN7HJYQG2GTP2Q4BMNDYMXBF2QTCUVRCWWXESDFKKWLMA, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.511231076 (55m3.550804186s) [noise=true , address=45.91.116.6:4975 , piece_id=2235T767SMWB6RAQHL3KO2WKQLM6I2YG3QNNZ4MXWDILETWXGDBQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.511452599 (55m3.551025709s) [noise=true , address=45.91.116.4:4938 , piece_id=VUPEQ3QWJD2SOESNOVPJFDTP2AVXXODL2YU2PNOWNY2WFE2LZIGA, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.511818941 (55m3.551392061s) [noise=true , address=45.91.116.13:4153 , piece_id=6XRM6HMSR6WVBE67ADFZ75SMYYNVSW2J5E4EWWPYUNAXDWJ47IKQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.512033562 (55m3.551606682s) [noise=true , address=45.91.116.8:4047 , piece_id=PGJFNH4LNUQQ5RIIUVSWE6BS2YTX5ZYEXZ4DGEVG2UZ5GV3JYZ3A, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.512388601 (55m3.551961721s) [noise=true , address=77.90.31.15:4699 , piece_id=7YD7B33QNRPFA7YBPVGW24ENCC3KAFA5BFOIL5NMPBIIGOQO6HEQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.512616898 (55m3.552190018s) [noise=true , address=77.90.31.18:4805 , piece_id=JE23EWPV7Q32QM7HUQP765OP7D2J3Q3MILLPJACHHBFF533OPFMQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.512814125 (55m3.552387245s) [noise=true , address=77.90.31.10:4517 , piece_id=G2A6WTX5WGNUPHWB7I3KW7X6XHNFND74AKP3CH5BCDMU7URZ4AOQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:27.512993268 (55m3.552566398s) [noise=true , address=45.91.116.7:4009 , piece_id=6RZZQPT5VE4GE65UDKUQJFSNZWF6GRORLJTP6TDRZUTB3KYF6MKQ, seg_pos={0 0}, upload=795]: Uploading piece... Sep 6 14:23:29.345773170 (55m5.38534631s) [upload=795]: Next returned nil segment Sep 6 14:23:29.345792267 (55m5.385365407s) [upload=795]: Waiting for error group managing segments... Sep 6 14:23:29.655937949 (55m5.695511069s) [noise=true , address=45.91.116.6:4975 , piece_id=2235T767SMWB6RAQHL3KO2WKQLM6I2YG3QNNZ4MXWDILETWXGDBQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.707154954 (55m5.746728064s) [noise=true , address=77.90.31.12:4590 , piece_id=4WZHDYEAN7HJYQG2GTP2Q4BMNDYMXBF2QTCUVRCWWXESDFKKWLMA, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.711780515 (55m5.751353635s) [noise=true , address=45.91.116.13:4153 , piece_id=6XRM6HMSR6WVBE67ADFZ75SMYYNVSW2J5E4EWWPYUNAXDWJ47IKQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.732003632 (55m5.771576752s) [noise=true , address=45.91.116.9:4262 , piece_id=QRIEQRJZ23HJIOUHRTZ277G7U6JSJZ4E4AQLFV67YP3PI2BLPFNQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.769201452 (55m5.808774572s) [noise=true , address=77.90.31.12:4592 , piece_id=ERYQTRSCQYS7I44QDFYZQZ6OIT5IIE2ESZHVYHEPQBWUZEXXZECA, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.774751672 (55m5.814324792s) [noise=true , address=77.90.31.14:4663 , piece_id=AT5XH7MX6UKIVAFAZD2BPMF55JFDVSQRSRCELU7W5AI6W7YS6NLQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.775558666 (55m5.815131786s) [noise=true , address=77.90.31.15:4699 , piece_id=7YD7B33QNRPFA7YBPVGW24ENCC3KAFA5BFOIL5NMPBIIGOQO6HEQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.777348020 (55m5.81692114s) [noise=true , address=45.91.116.4:4938 , piece_id=VUPEQ3QWJD2SOESNOVPJFDTP2AVXXODL2YU2PNOWNY2WFE2LZIGA, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.796849637 (55m5.836422747s) [noise=true , address=45.91.116.5:4297 , piece_id=ZY6QNRWYG2XN7J6RWLBDJDIQHU7QFIZFRMHKUJQ6ENHLKII352BQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.802265300 (55m5.84183842s) [noise=true , address=77.90.31.18:4805 , piece_id=JE23EWPV7Q32QM7HUQP765OP7D2J3Q3MILLPJACHHBFF533OPFMQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:29.820835765 (55m5.860408875s) [noise=true , address=45.91.116.9:4261 , piece_id=YLM3DM3MICEIRR2YK6ERCWAYC4UIVF53DIR2XJ4ZEWMTHEG3WSZQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:30.029044508 (55m6.068617618s) [noise=true , address=45.91.116.13:4154 , piece_id=YIX5O3FX7G2FN5TYV62DSA2VILKWB5WE7LMVQSHK6HNLPYSEH2GA, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:30.148837585 (55m6.188410705s) [noise=true , address=45.91.116.7:4009 , piece_id=6RZZQPT5VE4GE65UDKUQJFSNZWF6GRORLJTP6TDRZUTB3KYF6MKQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: <nil> Sep 6 14:23:30.148861891 (55m6.188435001s) [seg_pos={0 0}, upload=795]: Segment reached optimal threshold of 13 pieces. Sep 6 14:23:30.148950732 (55m6.188523852s) [noise=true , address=45.91.116.10:4083 , piece_id=QZHEMQ7EIQ6HV72IOSYWAM3QB44KS3VSBSMRWF7GYZLACWW4ZU3A, seg_pos={0 0}, upload=795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Sep 6 14:23:30.149024142 (55m6.188597262s) [noise=true , address=77.90.31.10:4517 , piece_id=G2A6WTX5WGNUPHWB7I3KW7X6XHNFND74AKP3CH5BCDMU7URZ4AOQ, seg_pos={0 0}, upload=795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: expected piece hash; context canceled; EOF Sep 6 14:23:30.149081912 (55m6.188655032s) [noise=true , address=45.91.116.8:4047 , piece_id=PGJFNH4LNUQQ5RIIUVSWE6BS2YTX5ZYEXZ4DGEVG2UZ5GV3JYZ3A, seg_pos={0 0}, upload=795]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:59152->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:59152->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59152->45.91.116.8:4047: use of closed network connection Sep 6 14:23:30.149097222 (55m6.188670342s) [seg_pos={0 0}, upload=795]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:23:30.149159821 (55m6.188732942s) [upload=795]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:23:30.149163969 (55m6.188737089s) [upload=795]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:23:30.575030106 (55m6.614603226s) [upload=795]: Done uploading segments... Sep 6 14:23:30.575056676 (55m6.614629837s) [upload=795]: Upload finished. err: <nil> Duration: 30.002686242s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X.X] duration: 30.002408341s FAIL (2025-09-06 14:23:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:22:57.031148012 (54m33.070721132s) [upload=789]: Starting upload Sep 6 14:22:57.031204260 (54m33.07077737s) [upload=789]: Done starting upload Sep 6 14:22:57.031278512 (54m33.070851632s) [upload=789]: Uploading segments... Sep 6 14:22:57.031423299 (54m33.070996419s) [upload=789]: Got next segment. Inline: false Sep 6 14:22:57.031454198 (54m33.071027398s) [upload=789]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:22:57.031464587 (54m33.071037797s) [upload=789]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:22:57.278698877 (54m33.318271997s) [seg_pos={0 0}, upload=789]: Begin upload segment... Sep 6 14:22:57.278745546 (54m33.318318666s) [seg_pos={0 0}, upload=789]: Done begin upload segment. Sep 6 14:22:57.278766046 (54m33.318339166s) [noise=true , address=77.90.31.21:4879 , piece_id=H5NZ7TZJ3UR6WLAUX7DG3PXPOOXBCDA3Z2KK3MWI72DSYIM6ISTA, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.278809398 (54m33.318382518s) [noise=true , address=77.90.31.17:4771 , piece_id=TUAVPREEY3F3LJXYNMHD3R63BY7XC2H55ZC36LGG22P7CFXTIPVA, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.279166692 (54m33.318739812s) [noise=true , address=77.90.31.10:4518 , piece_id=SEH7VJV72A5Y2XGWWUBFHZLVO43TV3TCAFMMHT3Y6XL6PXNIJM4A, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.279253458 (54m33.318826608s) [noise=true , address=45.91.116.5:4297 , piece_id=LXI66QOSF7PXCN3S32FY4DAOCUKUANS4MBAHTENHKUB4UZWN3VIQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.279469311 (54m33.319042431s) [noise=true , address=45.91.116.6:4975 , piece_id=BNKTBDKFDTPD7ANW5TDEJUWHCQUK6FZBKMCYSJPOXTOBBKHA3FLQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.279576987 (54m33.319150107s) [noise=true , address=77.90.31.21:4878 , piece_id=OIDN6FQNLHJSRV57PJ6ICY5EGXMRMG3HG4HLPBUGRH2M7I6QNQ4A, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.279861001 (54m33.319434121s) [noise=true , address=45.91.116.14:4191 , piece_id=32DKNDH7STRRHH5VM66IICTHRWVWHPQNA7N7OEGNBEFTGAMEAZ2Q, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.279874877 (54m33.319447997s) [noise=true , address=77.90.31.16:4733 , piece_id=ENAPA5TMKYVHC6WPDGNATTWIJD2BTBEIIYLPYPVSYXKOLQDO6PTA, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.280131489 (54m33.319704609s) [noise=true , address=45.91.116.6:4974 , piece_id=M6PLOXGOC3M5X2KOFESKAIUPIJQVOUHSKP3CCAQJUY2CV5T7R4MQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.280178538 (54m33.319751658s) [noise=true , address=45.91.116.13:4155 , piece_id=JE2AGWUC5HTHEEIXBRWYQXJTRJOSMKZC7L7RTA73XAD54JA64NHQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.280421423 (54m33.319994543s) [noise=true , address=45.91.116.7:4009 , piece_id=TWYT25CJRKDMNXQI7HBD4DYKPAMCH7QWGYDN7RSJPMZ3PPIKESBA, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.280551512 (54m33.320124632s) [noise=true , address=45.91.116.5:4299 , piece_id=EV3ZYSSVUXMIDWIBHXB4BTSYSJKXT2A7FZ6KGXMPHQXTYU5N6WXQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.280775070 (54m33.32034819s) [noise=true , address=45.91.116.12:4227 , piece_id=DXMEZTGTN2JBN7VLODJZVDCASDB5672PHHPO6REI6E7EQ43R2EKQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.280903947 (54m33.320477067s) [noise=true , address=45.91.116.4:4938 , piece_id=JRLO3OJV2HO6DHDYSIEERRHF7AP7EL2HPTUPCALO5HHVVXTQTWHA, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.281144467 (54m33.320717587s) [noise=true , address=45.91.116.10:4083 , piece_id=DAYR2UTKBVONEQWMKFMCRIBTAW44WBIYHBCAALSLP5V26JQPAW3A, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:57.281389145 (54m33.320962265s) [noise=true , address=45.91.116.4:4940 , piece_id=BN76USZFV72L6KL4MRV7QCE4FV3NT4AH73VMONBWWA74HECTOTWQ, seg_pos={0 0}, upload=789]: Uploading piece... Sep 6 14:22:59.133755998 (54m35.173329118s) [upload=789]: Next returned nil segment Sep 6 14:22:59.133771558 (54m35.173344668s) [upload=789]: Waiting for error group managing segments... Sep 6 14:22:59.439977661 (54m35.479550781s) [noise=true , address=45.91.116.5:4297 , piece_id=LXI66QOSF7PXCN3S32FY4DAOCUKUANS4MBAHTENHKUB4UZWN3VIQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.506696229 (54m35.546269349s) [noise=true , address=45.91.116.6:4974 , piece_id=M6PLOXGOC3M5X2KOFESKAIUPIJQVOUHSKP3CCAQJUY2CV5T7R4MQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.512005047 (54m35.551578167s) [noise=true , address=45.91.116.5:4299 , piece_id=EV3ZYSSVUXMIDWIBHXB4BTSYSJKXT2A7FZ6KGXMPHQXTYU5N6WXQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.563415302 (54m35.602988422s) [noise=true , address=45.91.116.4:4938 , piece_id=JRLO3OJV2HO6DHDYSIEERRHF7AP7EL2HPTUPCALO5HHVVXTQTWHA, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.564931503 (54m35.604504624s) [noise=true , address=45.91.116.12:4227 , piece_id=DXMEZTGTN2JBN7VLODJZVDCASDB5672PHHPO6REI6E7EQ43R2EKQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.569049173 (54m35.608622293s) [noise=true , address=77.90.31.10:4518 , piece_id=SEH7VJV72A5Y2XGWWUBFHZLVO43TV3TCAFMMHT3Y6XL6PXNIJM4A, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.575689239 (54m35.615262359s) [noise=true , address=45.91.116.6:4975 , piece_id=BNKTBDKFDTPD7ANW5TDEJUWHCQUK6FZBKMCYSJPOXTOBBKHA3FLQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.579823029 (54m35.619396139s) [noise=true , address=77.90.31.21:4879 , piece_id=H5NZ7TZJ3UR6WLAUX7DG3PXPOOXBCDA3Z2KK3MWI72DSYIM6ISTA, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.587420176 (54m35.626993296s) [noise=true , address=77.90.31.17:4771 , piece_id=TUAVPREEY3F3LJXYNMHD3R63BY7XC2H55ZC36LGG22P7CFXTIPVA, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.597623771 (54m35.637196891s) [noise=true , address=45.91.116.13:4155 , piece_id=JE2AGWUC5HTHEEIXBRWYQXJTRJOSMKZC7L7RTA73XAD54JA64NHQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.608563815 (54m35.648136935s) [noise=true , address=45.91.116.14:4191 , piece_id=32DKNDH7STRRHH5VM66IICTHRWVWHPQNA7N7OEGNBEFTGAMEAZ2Q, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.660887468 (54m35.700460578s) [noise=true , address=77.90.31.21:4878 , piece_id=OIDN6FQNLHJSRV57PJ6ICY5EGXMRMG3HG4HLPBUGRH2M7I6QNQ4A, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.718366686 (54m35.757939806s) [noise=true , address=45.91.116.4:4940 , piece_id=BN76USZFV72L6KL4MRV7QCE4FV3NT4AH73VMONBWWA74HECTOTWQ, seg_pos={0 0}, upload=789]: Done uploading piece. err: <nil> Sep 6 14:22:59.718397615 (54m35.757970735s) [seg_pos={0 0}, upload=789]: Segment reached optimal threshold of 13 pieces. Sep 6 14:22:59.718492657 (54m35.758065777s) [noise=true , address=45.91.116.7:4009 , piece_id=TWYT25CJRKDMNXQI7HBD4DYKPAMCH7QWGYDN7RSJPMZ3PPIKESBA, seg_pos={0 0}, upload=789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: expected piece hash; context canceled; EOF Sep 6 14:22:59.718623467 (54m35.758196577s) [noise=true , address=77.90.31.16:4733 , piece_id=ENAPA5TMKYVHC6WPDGNATTWIJD2BTBEIIYLPYPVSYXKOLQDO6PTA, seg_pos={0 0}, upload=789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12TBBdm1hfKVn49KrFVNj3MRbqoanGxJW82ws1iJqMtjRdzJKVf): protocol: expected piece hash; context canceled; EOF Sep 6 14:22:59.718673142 (54m35.758246262s) [noise=true , address=45.91.116.10:4083 , piece_id=DAYR2UTKBVONEQWMKFMCRIBTAW44WBIYHBCAALSLP5V26JQPAW3A, seg_pos={0 0}, upload=789]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:48382->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:48382->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48382->45.91.116.10:4083: use of closed network connection Sep 6 14:22:59.718692830 (54m35.75826595s) [seg_pos={0 0}, upload=789]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:22:59.718763806 (54m35.758336926s) [upload=789]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:22:59.718767774 (54m35.758340894s) [upload=789]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:23:00.028994901 (54m36.068568021s) [upload=789]: Done uploading segments... Sep 6 14:23:00.029013546 (54m36.068586666s) [upload=789]: Upload finished. err: <nil> Duration: 30.002435133s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [X........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X.] duration: 27.114121959s FAIL (2025-09-06 14:22:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:21:09.572244745 (52m45.611817865s) [upload=760]: Starting upload Sep 6 14:21:09.572309860 (52m45.61188297s) [upload=760]: Done starting upload Sep 6 14:21:09.572387397 (52m45.611960507s) [upload=760]: Uploading segments... Sep 6 14:21:09.572504272 (52m45.612077392s) [upload=760]: Got next segment. Inline: false Sep 6 14:21:09.572532656 (52m45.612105776s) [upload=760]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:21:09.572537034 (52m45.612110154s) [upload=760]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:21:09.823954247 (52m45.863527367s) [seg_pos={0 0}, upload=760]: Begin upload segment... Sep 6 14:21:09.824004293 (52m45.863577413s) [seg_pos={0 0}, upload=760]: Done begin upload segment. Sep 6 14:21:09.824052185 (52m45.863625305s) [noise=true , address=45.91.116.4:4938 , piece_id=QS4K26YPDPYOOOFPWJOZLF3ECOEEKAUFLMDRPOMGXH4OEZA3SXCQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824088233 (52m45.863661363s) [noise=true , address=45.91.116.6:4973 , piece_id=YNPTQ7NAPGNGIFQSVBX3ITSXHQIZVTLLXMGQMZMRKQV3X4ZULSBQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824410150 (52m45.86398327s) [noise=true , address=45.91.116.6:4974 , piece_id=Q63LAM5POLYFIO6FTKWL4IOVYNNNXM4FDDNU2ELHFR74UFKKPVRQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824658364 (52m45.864231475s) [noise=true , address=77.90.31.15:4697 , piece_id=BJ5G653G3JTSXT7HY5JM44HYKZZUT7WGZXTEMK3SV55VF4GOEHLA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824707679 (52m45.864280799s) [noise=true , address=45.91.116.14:4189 , piece_id=4TL553BTYDJIMHYZ3E3XICOIK24TMRVBNVLIWWAZ2XBDWAMQ5KTA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824971153 (52m45.864544263s) [noise=true , address=45.91.116.12:4226 , piece_id=DVIWIM3OG4D7WHV66DNHA7NW7IBPS5H5HBWCYSVYBYHJGUREN5YQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825183189 (52m45.864756309s) [noise=true , address=45.91.116.12:4227 , piece_id=TESPFBRDANBVPOQXCLIQ42BR4MHNFSNOXCJU5W43XEQTCHMU325A, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825232082 (52m45.864805202s) [noise=true , address=45.91.116.6:4975 , piece_id=LAWBIQ6TLZY3STFXFXJJLUB3N7INQNV5MNGX7HOOTT735TQWFYCQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825513761 (52m45.865086881s) [noise=true , address=77.90.31.12:4589 , piece_id=RWEXGA6NWBBSQM7L5BOD2YWQZ7B3CEHIF3NPHG2OFI3BVQWHCU6Q, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825687484 (52m45.865260604s) [noise=true , address=77.90.31.17:4771 , piece_id=UY6YK7CSB7HBUFXN2VXISPAIIM54SUP62OTQPZGNX4NKKODH7WEQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825726739 (52m45.865299859s) [noise=true , address=45.91.116.5:4298 , piece_id=BN6YEQDEZ4TZAAUMSPBYRDNZ2YSJXSDV3LNVBBJEU6F5UXEI5LYA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825989011 (52m45.865562131s) [noise=true , address=45.91.116.10:4083 , piece_id=ZWXVLA7FEEHJ2VA5XGBWKMWYVI5QATML45EPVAR7QJIMFAF4JKVQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.826205966 (52m45.865779086s) [noise=true , address=45.91.116.8:4048 , piece_id=O6RYY2N7HT2DTR6VYIG3QEBVO7DQQFAIEJT2P5R34675IG5WJQJQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.826646469 (52m45.866219589s) [noise=true , address=45.91.116.5:4297 , piece_id=PFLCJTQC3QU3FRNFKH4WAPSHBROMYCHT3MED3HGUFSF5RE4RLWDA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.828740235 (52m45.868313355s) [noise=true , address=77.90.31.10:4517 , piece_id=NMVJK3IPE2FLFF2WETWM6MB7ZUDZQM5AQYW5XLUOPRJTFQIJYYCQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.831676544 (52m45.871249674s) [noise=true , address=45.91.116.8:4045 , piece_id=W3C2K5OWH7OCGFACWDR2KCXCCFGT27GLAN4TEMGC2YFMODFZCE2Q, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:11.648442574 (52m47.688015694s) [upload=760]: Next returned nil segment Sep 6 14:21:11.648455980 (52m47.6880291s) [upload=760]: Waiting for error group managing segments... Sep 6 14:21:11.990076820 (52m48.02964993s) [noise=true , address=45.91.116.8:4045 , piece_id=W3C2K5OWH7OCGFACWDR2KCXCCFGT27GLAN4TEMGC2YFMODFZCE2Q, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.031669990 (52m48.07124312s) [noise=true , address=77.90.31.10:4517 , piece_id=NMVJK3IPE2FLFF2WETWM6MB7ZUDZQM5AQYW5XLUOPRJTFQIJYYCQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.031948002 (52m48.071521112s) [noise=true , address=45.91.116.6:4975 , piece_id=LAWBIQ6TLZY3STFXFXJJLUB3N7INQNV5MNGX7HOOTT735TQWFYCQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.048195191 (52m48.087768312s) [noise=true , address=45.91.116.4:4938 , piece_id=QS4K26YPDPYOOOFPWJOZLF3ECOEEKAUFLMDRPOMGXH4OEZA3SXCQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.050124352 (52m48.089697472s) [noise=true , address=45.91.116.6:4973 , piece_id=YNPTQ7NAPGNGIFQSVBX3ITSXHQIZVTLLXMGQMZMRKQV3X4ZULSBQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.055461426 (52m48.095034536s) [noise=true , address=45.91.116.10:4083 , piece_id=ZWXVLA7FEEHJ2VA5XGBWKMWYVI5QATML45EPVAR7QJIMFAF4JKVQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.083686826 (52m48.123259946s) [noise=true , address=45.91.116.8:4048 , piece_id=O6RYY2N7HT2DTR6VYIG3QEBVO7DQQFAIEJT2P5R34675IG5WJQJQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.105566312 (52m48.145139432s) [noise=true , address=45.91.116.12:4226 , piece_id=DVIWIM3OG4D7WHV66DNHA7NW7IBPS5H5HBWCYSVYBYHJGUREN5YQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.116671672 (52m48.156244792s) [noise=true , address=45.91.116.12:4227 , piece_id=TESPFBRDANBVPOQXCLIQ42BR4MHNFSNOXCJU5W43XEQTCHMU325A, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.151489097 (52m48.191062207s) [noise=true , address=77.90.31.12:4589 , piece_id=RWEXGA6NWBBSQM7L5BOD2YWQZ7B3CEHIF3NPHG2OFI3BVQWHCU6Q, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.163920784 (52m48.203493904s) [noise=true , address=45.91.116.14:4189 , piece_id=4TL553BTYDJIMHYZ3E3XICOIK24TMRVBNVLIWWAZ2XBDWAMQ5KTA, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.188451034 (52m48.228024374s) [noise=true , address=45.91.116.5:4297 , piece_id=PFLCJTQC3QU3FRNFKH4WAPSHBROMYCHT3MED3HGUFSF5RE4RLWDA, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.202325203 (52m48.241898313s) [noise=true , address=45.91.116.6:4974 , piece_id=Q63LAM5POLYFIO6FTKWL4IOVYNNNXM4FDDNU2ELHFR74UFKKPVRQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.202342106 (52m48.241915226s) [seg_pos={0 0}, upload=760]: Segment reached optimal threshold of 13 pieces. Sep 6 14:21:12.202457376 (52m48.242030496s) [noise=true , address=77.90.31.17:4771 , piece_id=UY6YK7CSB7HBUFXN2VXISPAIIM54SUP62OTQPZGNX4NKKODH7WEQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Sep 6 14:21:12.202519174 (52m48.242092284s) [noise=true , address=77.90.31.15:4697 , piece_id=BJ5G653G3JTSXT7HY5JM44HYKZZUT7WGZXTEMK3SV55VF4GOEHLA, seg_pos={0 0}, upload=760]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: read tcp 188.166.225.170:52486->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 14:21:12.202572837 (52m48.242145957s) [noise=true , address=45.91.116.5:4298 , piece_id=BN6YEQDEZ4TZAAUMSPBYRDNZ2YSJXSDV3LNVBBJEU6F5UXEI5LYA, seg_pos={0 0}, upload=760]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: read tcp 188.166.225.170:42188->45.91.116.5:4298: use of closed network connection; context canceled Sep 6 14:21:12.202583447 (52m48.242156567s) [seg_pos={0 0}, upload=760]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:21:12.202643643 (52m48.242216763s) [upload=760]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:21:12.202647530 (52m48.24222065s) [upload=760]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:21:12.506161912 (52m48.545735032s) [upload=760]: Done uploading segments... Sep 6 14:21:12.506180989 (52m48.545754109s) [upload=760]: Upload finished. err: <nil> Duration: 27.114149132s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 26 out of 60 is failed (threshold=3) [XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..X] duration: 30.003538262s FAIL (2025-09-06 14:21:38 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:21:09.572244745 (52m45.611817865s) [upload=760]: Starting upload Sep 6 14:21:09.572309860 (52m45.61188297s) [upload=760]: Done starting upload Sep 6 14:21:09.572387397 (52m45.611960507s) [upload=760]: Uploading segments... Sep 6 14:21:09.572504272 (52m45.612077392s) [upload=760]: Got next segment. Inline: false Sep 6 14:21:09.572532656 (52m45.612105776s) [upload=760]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:21:09.572537034 (52m45.612110154s) [upload=760]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:21:09.823954247 (52m45.863527367s) [seg_pos={0 0}, upload=760]: Begin upload segment... Sep 6 14:21:09.824004293 (52m45.863577413s) [seg_pos={0 0}, upload=760]: Done begin upload segment. Sep 6 14:21:09.824052185 (52m45.863625305s) [noise=true , address=45.91.116.4:4938 , piece_id=QS4K26YPDPYOOOFPWJOZLF3ECOEEKAUFLMDRPOMGXH4OEZA3SXCQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824088233 (52m45.863661363s) [noise=true , address=45.91.116.6:4973 , piece_id=YNPTQ7NAPGNGIFQSVBX3ITSXHQIZVTLLXMGQMZMRKQV3X4ZULSBQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824410150 (52m45.86398327s) [noise=true , address=45.91.116.6:4974 , piece_id=Q63LAM5POLYFIO6FTKWL4IOVYNNNXM4FDDNU2ELHFR74UFKKPVRQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824658364 (52m45.864231475s) [noise=true , address=77.90.31.15:4697 , piece_id=BJ5G653G3JTSXT7HY5JM44HYKZZUT7WGZXTEMK3SV55VF4GOEHLA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824707679 (52m45.864280799s) [noise=true , address=45.91.116.14:4189 , piece_id=4TL553BTYDJIMHYZ3E3XICOIK24TMRVBNVLIWWAZ2XBDWAMQ5KTA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.824971153 (52m45.864544263s) [noise=true , address=45.91.116.12:4226 , piece_id=DVIWIM3OG4D7WHV66DNHA7NW7IBPS5H5HBWCYSVYBYHJGUREN5YQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825183189 (52m45.864756309s) [noise=true , address=45.91.116.12:4227 , piece_id=TESPFBRDANBVPOQXCLIQ42BR4MHNFSNOXCJU5W43XEQTCHMU325A, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825232082 (52m45.864805202s) [noise=true , address=45.91.116.6:4975 , piece_id=LAWBIQ6TLZY3STFXFXJJLUB3N7INQNV5MNGX7HOOTT735TQWFYCQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825513761 (52m45.865086881s) [noise=true , address=77.90.31.12:4589 , piece_id=RWEXGA6NWBBSQM7L5BOD2YWQZ7B3CEHIF3NPHG2OFI3BVQWHCU6Q, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825687484 (52m45.865260604s) [noise=true , address=77.90.31.17:4771 , piece_id=UY6YK7CSB7HBUFXN2VXISPAIIM54SUP62OTQPZGNX4NKKODH7WEQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825726739 (52m45.865299859s) [noise=true , address=45.91.116.5:4298 , piece_id=BN6YEQDEZ4TZAAUMSPBYRDNZ2YSJXSDV3LNVBBJEU6F5UXEI5LYA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.825989011 (52m45.865562131s) [noise=true , address=45.91.116.10:4083 , piece_id=ZWXVLA7FEEHJ2VA5XGBWKMWYVI5QATML45EPVAR7QJIMFAF4JKVQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.826205966 (52m45.865779086s) [noise=true , address=45.91.116.8:4048 , piece_id=O6RYY2N7HT2DTR6VYIG3QEBVO7DQQFAIEJT2P5R34675IG5WJQJQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.826646469 (52m45.866219589s) [noise=true , address=45.91.116.5:4297 , piece_id=PFLCJTQC3QU3FRNFKH4WAPSHBROMYCHT3MED3HGUFSF5RE4RLWDA, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.828740235 (52m45.868313355s) [noise=true , address=77.90.31.10:4517 , piece_id=NMVJK3IPE2FLFF2WETWM6MB7ZUDZQM5AQYW5XLUOPRJTFQIJYYCQ, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:09.831676544 (52m45.871249674s) [noise=true , address=45.91.116.8:4045 , piece_id=W3C2K5OWH7OCGFACWDR2KCXCCFGT27GLAN4TEMGC2YFMODFZCE2Q, seg_pos={0 0}, upload=760]: Uploading piece... Sep 6 14:21:11.648442574 (52m47.688015694s) [upload=760]: Next returned nil segment Sep 6 14:21:11.648455980 (52m47.6880291s) [upload=760]: Waiting for error group managing segments... Sep 6 14:21:11.990076820 (52m48.02964993s) [noise=true , address=45.91.116.8:4045 , piece_id=W3C2K5OWH7OCGFACWDR2KCXCCFGT27GLAN4TEMGC2YFMODFZCE2Q, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.031669990 (52m48.07124312s) [noise=true , address=77.90.31.10:4517 , piece_id=NMVJK3IPE2FLFF2WETWM6MB7ZUDZQM5AQYW5XLUOPRJTFQIJYYCQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.031948002 (52m48.071521112s) [noise=true , address=45.91.116.6:4975 , piece_id=LAWBIQ6TLZY3STFXFXJJLUB3N7INQNV5MNGX7HOOTT735TQWFYCQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.048195191 (52m48.087768312s) [noise=true , address=45.91.116.4:4938 , piece_id=QS4K26YPDPYOOOFPWJOZLF3ECOEEKAUFLMDRPOMGXH4OEZA3SXCQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.050124352 (52m48.089697472s) [noise=true , address=45.91.116.6:4973 , piece_id=YNPTQ7NAPGNGIFQSVBX3ITSXHQIZVTLLXMGQMZMRKQV3X4ZULSBQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.055461426 (52m48.095034536s) [noise=true , address=45.91.116.10:4083 , piece_id=ZWXVLA7FEEHJ2VA5XGBWKMWYVI5QATML45EPVAR7QJIMFAF4JKVQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.083686826 (52m48.123259946s) [noise=true , address=45.91.116.8:4048 , piece_id=O6RYY2N7HT2DTR6VYIG3QEBVO7DQQFAIEJT2P5R34675IG5WJQJQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.105566312 (52m48.145139432s) [noise=true , address=45.91.116.12:4226 , piece_id=DVIWIM3OG4D7WHV66DNHA7NW7IBPS5H5HBWCYSVYBYHJGUREN5YQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.116671672 (52m48.156244792s) [noise=true , address=45.91.116.12:4227 , piece_id=TESPFBRDANBVPOQXCLIQ42BR4MHNFSNOXCJU5W43XEQTCHMU325A, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.151489097 (52m48.191062207s) [noise=true , address=77.90.31.12:4589 , piece_id=RWEXGA6NWBBSQM7L5BOD2YWQZ7B3CEHIF3NPHG2OFI3BVQWHCU6Q, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.163920784 (52m48.203493904s) [noise=true , address=45.91.116.14:4189 , piece_id=4TL553BTYDJIMHYZ3E3XICOIK24TMRVBNVLIWWAZ2XBDWAMQ5KTA, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.188451034 (52m48.228024374s) [noise=true , address=45.91.116.5:4297 , piece_id=PFLCJTQC3QU3FRNFKH4WAPSHBROMYCHT3MED3HGUFSF5RE4RLWDA, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.202325203 (52m48.241898313s) [noise=true , address=45.91.116.6:4974 , piece_id=Q63LAM5POLYFIO6FTKWL4IOVYNNNXM4FDDNU2ELHFR74UFKKPVRQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: <nil> Sep 6 14:21:12.202342106 (52m48.241915226s) [seg_pos={0 0}, upload=760]: Segment reached optimal threshold of 13 pieces. Sep 6 14:21:12.202457376 (52m48.242030496s) [noise=true , address=77.90.31.17:4771 , piece_id=UY6YK7CSB7HBUFXN2VXISPAIIM54SUP62OTQPZGNX4NKKODH7WEQ, seg_pos={0 0}, upload=760]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Sep 6 14:21:12.202519174 (52m48.242092284s) [noise=true , address=77.90.31.15:4697 , piece_id=BJ5G653G3JTSXT7HY5JM44HYKZZUT7WGZXTEMK3SV55VF4GOEHLA, seg_pos={0 0}, upload=760]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: read tcp 188.166.225.170:52486->77.90.31.15:4697: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 14:21:12.202572837 (52m48.242145957s) [noise=true , address=45.91.116.5:4298 , piece_id=BN6YEQDEZ4TZAAUMSPBYRDNZ2YSJXSDV3LNVBBJEU6F5UXEI5LYA, seg_pos={0 0}, upload=760]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C): protocol: read tcp 188.166.225.170:42188->45.91.116.5:4298: use of closed network connection; context canceled Sep 6 14:21:12.202583447 (52m48.242156567s) [seg_pos={0 0}, upload=760]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:21:12.202643643 (52m48.242216763s) [upload=760]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:21:12.202647530 (52m48.24222065s) [upload=760]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:21:12.506161912 (52m48.545735032s) [upload=760]: Done uploading segments... Sep 6 14:21:12.506180989 (52m48.545754109s) [upload=760]: Upload finished. err: <nil> Duration: 30.003550736s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [.XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX..] duration: 27.579508848s FAIL (2025-09-06 14:20:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:18:48.057874731 (50m24.097447851s) [upload=724]: Starting upload Sep 6 14:18:48.057937511 (50m24.097510631s) [upload=724]: Done starting upload Sep 6 14:18:48.058060516 (50m24.097633706s) [upload=724]: Uploading segments... Sep 6 14:18:48.058133997 (50m24.097707117s) [upload=724]: Got next segment. Inline: false Sep 6 14:18:48.058164345 (50m24.097737465s) [upload=724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:18:48.058168814 (50m24.097741924s) [upload=724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:18:48.303956645 (50m24.343529765s) [seg_pos={0 0}, upload=724]: Begin upload segment... Sep 6 14:18:48.304000169 (50m24.343573289s) [seg_pos={0 0}, upload=724]: Done begin upload segment. Sep 6 14:18:48.304038642 (50m24.343611762s) [noise=true , address=45.91.116.11:4117 , piece_id=JQKTLMVS53ZOYDFUYCNFOGR6I34HVMDDMIXEMNUUNQGLK7DDPWYA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304105229 (50m24.343678359s) [noise=true , address=45.91.116.8:4045 , piece_id=62B2N4XGGDURHMDFGEOWD6YPKF2FCNWCAGVGADVJQHGHITTRI2ZQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304741968 (50m24.344315098s) [noise=true , address=77.90.31.19:4843 , piece_id=6AT5WONXRHAUNTJHJQNVMIMAJM3CWCOG7LGDXIVNBVE4TG2MQYCA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304773298 (50m24.344346418s) [noise=true , address=77.90.31.13:4627 , piece_id=YJZBHPHT2NJ7LN4W4N3UTC3VZM6CQWSSD4HXDBCAEGZCJZEXNBPA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305297571 (50m24.344870691s) [noise=true , address=77.90.31.18:4806 , piece_id=BHEMK7FEB2RI33OHORIXKOM3CU7BJVCKIA4D4SLKP3AEL5SDX4MA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305385088 (50m24.344958208s) [noise=true , address=45.91.116.13:4153 , piece_id=2OIOXIU2MGF32CB47QEJMAN7EXK2DA67BJG5RXG2MYDBQZ4HTBSQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305634395 (50m24.345207515s) [noise=true , address=45.91.116.14:4190 , piece_id=EAJ2ZHXYPAHMPWYJVVCAX22DRXADPGJBZK2Z4BL2C7CNOEKMD3BA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305846471 (50m24.345419591s) [noise=true , address=45.91.116.12:4226 , piece_id=XGA2YVX57U2ITAJGKDBVVGWS3XAT3BMOPMSGLFKRVRLCMRCZNX6A, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305951022 (50m24.345524142s) [noise=true , address=45.91.116.6:4973 , piece_id=GWRWUX6QPGIWNCQRPIITG7ST5SW4CF3ABSJ2SOH65XZKROHQU35Q, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306106750 (50m24.34567987s) [noise=true , address=45.91.116.14:4189 , piece_id=RZHOZRM3BBEBTLISYBZRFU7Y3ZDVO2KKEBNWI7HFYRO5N2T5DVIQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306244814 (50m24.345817934s) [noise=true , address=45.91.116.10:4081 , piece_id=SSOGHSJBNFJXEO5F5GFDRXZOLMVOZ76UBFHPHMO4VE4TGFMW5IOQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306330658 (50m24.345903778s) [noise=true , address=45.91.116.10:4082 , piece_id=XVHFGF3W2OBUEKMR72XQDWUZ6WO6LKBZHS2LVTXE3B2A2DGYQKRA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306578462 (50m24.346151582s) [noise=true , address=45.91.116.4:4937 , piece_id=PO2D7I75PT2B2JZAKMTQ322WI7OU5I77QBW7EZDI7WU5DLSS7ECQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306812089 (50m24.346385209s) [noise=true , address=45.91.116.12:4225 , piece_id=4ZCQ536QONPUNNHTRARFZ43T7NKUBRMMX6XRJ3GGEG7RXTHCXCUA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.307019446 (50m24.346592566s) [noise=true , address=45.91.116.13:4154 , piece_id=62WP2G7VG6JN4JQFLX5DNXTNU2NZDLOZHPSXKKI6YRTQTOMX5PCQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.307096814 (50m24.346669934s) [noise=true , address=45.91.116.5:4297 , piece_id=LQOF67PVBVTVBZQ5QRHNBF5WVNNQBMTZ4AQV4W3L356ZI5CJHXYA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:50.159084813 (50m26.198657933s) [upload=724]: Next returned nil segment Sep 6 14:18:50.159102357 (50m26.198675467s) [upload=724]: Waiting for error group managing segments... Sep 6 14:18:50.470774459 (50m26.510347579s) [noise=true , address=77.90.31.19:4843 , piece_id=6AT5WONXRHAUNTJHJQNVMIMAJM3CWCOG7LGDXIVNBVE4TG2MQYCA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.481408437 (50m26.520981557s) [noise=true , address=45.91.116.4:4937 , piece_id=PO2D7I75PT2B2JZAKMTQ322WI7OU5I77QBW7EZDI7WU5DLSS7ECQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.496055585 (50m26.535628705s) [noise=true , address=45.91.116.8:4045 , piece_id=62B2N4XGGDURHMDFGEOWD6YPKF2FCNWCAGVGADVJQHGHITTRI2ZQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.498101229 (50m26.537674349s) [noise=true , address=45.91.116.11:4117 , piece_id=JQKTLMVS53ZOYDFUYCNFOGR6I34HVMDDMIXEMNUUNQGLK7DDPWYA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.501428606 (50m26.541001726s) [noise=true , address=45.91.116.5:4297 , piece_id=LQOF67PVBVTVBZQ5QRHNBF5WVNNQBMTZ4AQV4W3L356ZI5CJHXYA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.505258865 (50m26.544831975s) [noise=true , address=45.91.116.12:4225 , piece_id=4ZCQ536QONPUNNHTRARFZ43T7NKUBRMMX6XRJ3GGEG7RXTHCXCUA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.512713841 (50m26.552286961s) [noise=true , address=45.91.116.13:4153 , piece_id=2OIOXIU2MGF32CB47QEJMAN7EXK2DA67BJG5RXG2MYDBQZ4HTBSQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.519686393 (50m26.559259513s) [noise=true , address=45.91.116.14:4190 , piece_id=EAJ2ZHXYPAHMPWYJVVCAX22DRXADPGJBZK2Z4BL2C7CNOEKMD3BA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.521989810 (50m26.56156293s) [noise=true , address=45.91.116.10:4082 , piece_id=XVHFGF3W2OBUEKMR72XQDWUZ6WO6LKBZHS2LVTXE3B2A2DGYQKRA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.522815411 (50m26.562388531s) [noise=true , address=45.91.116.10:4081 , piece_id=SSOGHSJBNFJXEO5F5GFDRXZOLMVOZ76UBFHPHMO4VE4TGFMW5IOQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.542621459 (50m26.58219458s) [noise=true , address=45.91.116.6:4973 , piece_id=GWRWUX6QPGIWNCQRPIITG7ST5SW4CF3ABSJ2SOH65XZKROHQU35Q, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.545162883 (50m26.584736003s) [noise=true , address=45.91.116.13:4154 , piece_id=62WP2G7VG6JN4JQFLX5DNXTNU2NZDLOZHPSXKKI6YRTQTOMX5PCQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.610468787 (50m26.650041907s) [noise=true , address=45.91.116.12:4226 , piece_id=XGA2YVX57U2ITAJGKDBVVGWS3XAT3BMOPMSGLFKRVRLCMRCZNX6A, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.610494627 (50m26.650067737s) [seg_pos={0 0}, upload=724]: Segment reached optimal threshold of 13 pieces. Sep 6 14:18:50.610700922 (50m26.650274042s) [noise=true , address=77.90.31.13:4627 , piece_id=YJZBHPHT2NJ7LN4W4N3UTC3VZM6CQWSSD4HXDBCAEGZCJZEXNBPA, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection Sep 6 14:18:50.610712313 (50m26.650285433s) [noise=true , address=45.91.116.14:4189 , piece_id=RZHOZRM3BBEBTLISYBZRFU7Y3ZDVO2KKEBNWI7HFYRO5N2T5DVIQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection Sep 6 14:18:50.610718085 (50m26.650291195s) [noise=true , address=77.90.31.18:4806 , piece_id=BHEMK7FEB2RI33OHORIXKOM3CU7BJVCKIA4D4SLKP3AEL5SDX4MA, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF Sep 6 14:18:50.610735127 (50m26.650308247s) [seg_pos={0 0}, upload=724]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:18:50.610802797 (50m26.650375917s) [upload=724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:18:50.610807045 (50m26.650380165s) [upload=724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:18:50.912618780 (50m26.9521919s) [upload=724]: Done uploading segments... Sep 6 14:18:50.912637626 (50m26.952210746s) [upload=724]: Upload finished. err: <nil> Duration: 27.579538195s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [..XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX.] duration: 19.771899594s FAIL (2025-09-06 14:19:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:18:48.057874731 (50m24.097447851s) [upload=724]: Starting upload Sep 6 14:18:48.057937511 (50m24.097510631s) [upload=724]: Done starting upload Sep 6 14:18:48.058060516 (50m24.097633706s) [upload=724]: Uploading segments... Sep 6 14:18:48.058133997 (50m24.097707117s) [upload=724]: Got next segment. Inline: false Sep 6 14:18:48.058164345 (50m24.097737465s) [upload=724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:18:48.058168814 (50m24.097741924s) [upload=724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:18:48.303956645 (50m24.343529765s) [seg_pos={0 0}, upload=724]: Begin upload segment... Sep 6 14:18:48.304000169 (50m24.343573289s) [seg_pos={0 0}, upload=724]: Done begin upload segment. Sep 6 14:18:48.304038642 (50m24.343611762s) [noise=true , address=45.91.116.11:4117 , piece_id=JQKTLMVS53ZOYDFUYCNFOGR6I34HVMDDMIXEMNUUNQGLK7DDPWYA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304105229 (50m24.343678359s) [noise=true , address=45.91.116.8:4045 , piece_id=62B2N4XGGDURHMDFGEOWD6YPKF2FCNWCAGVGADVJQHGHITTRI2ZQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304741968 (50m24.344315098s) [noise=true , address=77.90.31.19:4843 , piece_id=6AT5WONXRHAUNTJHJQNVMIMAJM3CWCOG7LGDXIVNBVE4TG2MQYCA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304773298 (50m24.344346418s) [noise=true , address=77.90.31.13:4627 , piece_id=YJZBHPHT2NJ7LN4W4N3UTC3VZM6CQWSSD4HXDBCAEGZCJZEXNBPA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305297571 (50m24.344870691s) [noise=true , address=77.90.31.18:4806 , piece_id=BHEMK7FEB2RI33OHORIXKOM3CU7BJVCKIA4D4SLKP3AEL5SDX4MA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305385088 (50m24.344958208s) [noise=true , address=45.91.116.13:4153 , piece_id=2OIOXIU2MGF32CB47QEJMAN7EXK2DA67BJG5RXG2MYDBQZ4HTBSQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305634395 (50m24.345207515s) [noise=true , address=45.91.116.14:4190 , piece_id=EAJ2ZHXYPAHMPWYJVVCAX22DRXADPGJBZK2Z4BL2C7CNOEKMD3BA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305846471 (50m24.345419591s) [noise=true , address=45.91.116.12:4226 , piece_id=XGA2YVX57U2ITAJGKDBVVGWS3XAT3BMOPMSGLFKRVRLCMRCZNX6A, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305951022 (50m24.345524142s) [noise=true , address=45.91.116.6:4973 , piece_id=GWRWUX6QPGIWNCQRPIITG7ST5SW4CF3ABSJ2SOH65XZKROHQU35Q, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306106750 (50m24.34567987s) [noise=true , address=45.91.116.14:4189 , piece_id=RZHOZRM3BBEBTLISYBZRFU7Y3ZDVO2KKEBNWI7HFYRO5N2T5DVIQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306244814 (50m24.345817934s) [noise=true , address=45.91.116.10:4081 , piece_id=SSOGHSJBNFJXEO5F5GFDRXZOLMVOZ76UBFHPHMO4VE4TGFMW5IOQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306330658 (50m24.345903778s) [noise=true , address=45.91.116.10:4082 , piece_id=XVHFGF3W2OBUEKMR72XQDWUZ6WO6LKBZHS2LVTXE3B2A2DGYQKRA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306578462 (50m24.346151582s) [noise=true , address=45.91.116.4:4937 , piece_id=PO2D7I75PT2B2JZAKMTQ322WI7OU5I77QBW7EZDI7WU5DLSS7ECQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306812089 (50m24.346385209s) [noise=true , address=45.91.116.12:4225 , piece_id=4ZCQ536QONPUNNHTRARFZ43T7NKUBRMMX6XRJ3GGEG7RXTHCXCUA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.307019446 (50m24.346592566s) [noise=true , address=45.91.116.13:4154 , piece_id=62WP2G7VG6JN4JQFLX5DNXTNU2NZDLOZHPSXKKI6YRTQTOMX5PCQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.307096814 (50m24.346669934s) [noise=true , address=45.91.116.5:4297 , piece_id=LQOF67PVBVTVBZQ5QRHNBF5WVNNQBMTZ4AQV4W3L356ZI5CJHXYA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:50.159084813 (50m26.198657933s) [upload=724]: Next returned nil segment Sep 6 14:18:50.159102357 (50m26.198675467s) [upload=724]: Waiting for error group managing segments... Sep 6 14:18:50.470774459 (50m26.510347579s) [noise=true , address=77.90.31.19:4843 , piece_id=6AT5WONXRHAUNTJHJQNVMIMAJM3CWCOG7LGDXIVNBVE4TG2MQYCA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.481408437 (50m26.520981557s) [noise=true , address=45.91.116.4:4937 , piece_id=PO2D7I75PT2B2JZAKMTQ322WI7OU5I77QBW7EZDI7WU5DLSS7ECQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.496055585 (50m26.535628705s) [noise=true , address=45.91.116.8:4045 , piece_id=62B2N4XGGDURHMDFGEOWD6YPKF2FCNWCAGVGADVJQHGHITTRI2ZQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.498101229 (50m26.537674349s) [noise=true , address=45.91.116.11:4117 , piece_id=JQKTLMVS53ZOYDFUYCNFOGR6I34HVMDDMIXEMNUUNQGLK7DDPWYA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.501428606 (50m26.541001726s) [noise=true , address=45.91.116.5:4297 , piece_id=LQOF67PVBVTVBZQ5QRHNBF5WVNNQBMTZ4AQV4W3L356ZI5CJHXYA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.505258865 (50m26.544831975s) [noise=true , address=45.91.116.12:4225 , piece_id=4ZCQ536QONPUNNHTRARFZ43T7NKUBRMMX6XRJ3GGEG7RXTHCXCUA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.512713841 (50m26.552286961s) [noise=true , address=45.91.116.13:4153 , piece_id=2OIOXIU2MGF32CB47QEJMAN7EXK2DA67BJG5RXG2MYDBQZ4HTBSQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.519686393 (50m26.559259513s) [noise=true , address=45.91.116.14:4190 , piece_id=EAJ2ZHXYPAHMPWYJVVCAX22DRXADPGJBZK2Z4BL2C7CNOEKMD3BA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.521989810 (50m26.56156293s) [noise=true , address=45.91.116.10:4082 , piece_id=XVHFGF3W2OBUEKMR72XQDWUZ6WO6LKBZHS2LVTXE3B2A2DGYQKRA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.522815411 (50m26.562388531s) [noise=true , address=45.91.116.10:4081 , piece_id=SSOGHSJBNFJXEO5F5GFDRXZOLMVOZ76UBFHPHMO4VE4TGFMW5IOQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.542621459 (50m26.58219458s) [noise=true , address=45.91.116.6:4973 , piece_id=GWRWUX6QPGIWNCQRPIITG7ST5SW4CF3ABSJ2SOH65XZKROHQU35Q, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.545162883 (50m26.584736003s) [noise=true , address=45.91.116.13:4154 , piece_id=62WP2G7VG6JN4JQFLX5DNXTNU2NZDLOZHPSXKKI6YRTQTOMX5PCQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.610468787 (50m26.650041907s) [noise=true , address=45.91.116.12:4226 , piece_id=XGA2YVX57U2ITAJGKDBVVGWS3XAT3BMOPMSGLFKRVRLCMRCZNX6A, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.610494627 (50m26.650067737s) [seg_pos={0 0}, upload=724]: Segment reached optimal threshold of 13 pieces. Sep 6 14:18:50.610700922 (50m26.650274042s) [noise=true , address=77.90.31.13:4627 , piece_id=YJZBHPHT2NJ7LN4W4N3UTC3VZM6CQWSSD4HXDBCAEGZCJZEXNBPA, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection Sep 6 14:18:50.610712313 (50m26.650285433s) [noise=true , address=45.91.116.14:4189 , piece_id=RZHOZRM3BBEBTLISYBZRFU7Y3ZDVO2KKEBNWI7HFYRO5N2T5DVIQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection Sep 6 14:18:50.610718085 (50m26.650291195s) [noise=true , address=77.90.31.18:4806 , piece_id=BHEMK7FEB2RI33OHORIXKOM3CU7BJVCKIA4D4SLKP3AEL5SDX4MA, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF Sep 6 14:18:50.610735127 (50m26.650308247s) [seg_pos={0 0}, upload=724]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:18:50.610802797 (50m26.650375917s) [upload=724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:18:50.610807045 (50m26.650380165s) [upload=724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:18:50.912618780 (50m26.9521919s) [upload=724]: Done uploading segments... Sep 6 14:18:50.912637626 (50m26.952210746s) [upload=724]: Upload finished. err: <nil> Duration: 19.771922207s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 25 out of 60 is failed (threshold=3) [...XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXXX] duration: 30.002044809s FAIL (2025-09-06 14:19:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:18:48.057874731 (50m24.097447851s) [upload=724]: Starting upload Sep 6 14:18:48.057937511 (50m24.097510631s) [upload=724]: Done starting upload Sep 6 14:18:48.058060516 (50m24.097633706s) [upload=724]: Uploading segments... Sep 6 14:18:48.058133997 (50m24.097707117s) [upload=724]: Got next segment. Inline: false Sep 6 14:18:48.058164345 (50m24.097737465s) [upload=724]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:18:48.058168814 (50m24.097741924s) [upload=724]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:18:48.303956645 (50m24.343529765s) [seg_pos={0 0}, upload=724]: Begin upload segment... Sep 6 14:18:48.304000169 (50m24.343573289s) [seg_pos={0 0}, upload=724]: Done begin upload segment. Sep 6 14:18:48.304038642 (50m24.343611762s) [noise=true , address=45.91.116.11:4117 , piece_id=JQKTLMVS53ZOYDFUYCNFOGR6I34HVMDDMIXEMNUUNQGLK7DDPWYA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304105229 (50m24.343678359s) [noise=true , address=45.91.116.8:4045 , piece_id=62B2N4XGGDURHMDFGEOWD6YPKF2FCNWCAGVGADVJQHGHITTRI2ZQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304741968 (50m24.344315098s) [noise=true , address=77.90.31.19:4843 , piece_id=6AT5WONXRHAUNTJHJQNVMIMAJM3CWCOG7LGDXIVNBVE4TG2MQYCA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.304773298 (50m24.344346418s) [noise=true , address=77.90.31.13:4627 , piece_id=YJZBHPHT2NJ7LN4W4N3UTC3VZM6CQWSSD4HXDBCAEGZCJZEXNBPA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305297571 (50m24.344870691s) [noise=true , address=77.90.31.18:4806 , piece_id=BHEMK7FEB2RI33OHORIXKOM3CU7BJVCKIA4D4SLKP3AEL5SDX4MA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305385088 (50m24.344958208s) [noise=true , address=45.91.116.13:4153 , piece_id=2OIOXIU2MGF32CB47QEJMAN7EXK2DA67BJG5RXG2MYDBQZ4HTBSQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305634395 (50m24.345207515s) [noise=true , address=45.91.116.14:4190 , piece_id=EAJ2ZHXYPAHMPWYJVVCAX22DRXADPGJBZK2Z4BL2C7CNOEKMD3BA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305846471 (50m24.345419591s) [noise=true , address=45.91.116.12:4226 , piece_id=XGA2YVX57U2ITAJGKDBVVGWS3XAT3BMOPMSGLFKRVRLCMRCZNX6A, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.305951022 (50m24.345524142s) [noise=true , address=45.91.116.6:4973 , piece_id=GWRWUX6QPGIWNCQRPIITG7ST5SW4CF3ABSJ2SOH65XZKROHQU35Q, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306106750 (50m24.34567987s) [noise=true , address=45.91.116.14:4189 , piece_id=RZHOZRM3BBEBTLISYBZRFU7Y3ZDVO2KKEBNWI7HFYRO5N2T5DVIQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306244814 (50m24.345817934s) [noise=true , address=45.91.116.10:4081 , piece_id=SSOGHSJBNFJXEO5F5GFDRXZOLMVOZ76UBFHPHMO4VE4TGFMW5IOQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306330658 (50m24.345903778s) [noise=true , address=45.91.116.10:4082 , piece_id=XVHFGF3W2OBUEKMR72XQDWUZ6WO6LKBZHS2LVTXE3B2A2DGYQKRA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306578462 (50m24.346151582s) [noise=true , address=45.91.116.4:4937 , piece_id=PO2D7I75PT2B2JZAKMTQ322WI7OU5I77QBW7EZDI7WU5DLSS7ECQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.306812089 (50m24.346385209s) [noise=true , address=45.91.116.12:4225 , piece_id=4ZCQ536QONPUNNHTRARFZ43T7NKUBRMMX6XRJ3GGEG7RXTHCXCUA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.307019446 (50m24.346592566s) [noise=true , address=45.91.116.13:4154 , piece_id=62WP2G7VG6JN4JQFLX5DNXTNU2NZDLOZHPSXKKI6YRTQTOMX5PCQ, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:48.307096814 (50m24.346669934s) [noise=true , address=45.91.116.5:4297 , piece_id=LQOF67PVBVTVBZQ5QRHNBF5WVNNQBMTZ4AQV4W3L356ZI5CJHXYA, seg_pos={0 0}, upload=724]: Uploading piece... Sep 6 14:18:50.159084813 (50m26.198657933s) [upload=724]: Next returned nil segment Sep 6 14:18:50.159102357 (50m26.198675467s) [upload=724]: Waiting for error group managing segments... Sep 6 14:18:50.470774459 (50m26.510347579s) [noise=true , address=77.90.31.19:4843 , piece_id=6AT5WONXRHAUNTJHJQNVMIMAJM3CWCOG7LGDXIVNBVE4TG2MQYCA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.481408437 (50m26.520981557s) [noise=true , address=45.91.116.4:4937 , piece_id=PO2D7I75PT2B2JZAKMTQ322WI7OU5I77QBW7EZDI7WU5DLSS7ECQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.496055585 (50m26.535628705s) [noise=true , address=45.91.116.8:4045 , piece_id=62B2N4XGGDURHMDFGEOWD6YPKF2FCNWCAGVGADVJQHGHITTRI2ZQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.498101229 (50m26.537674349s) [noise=true , address=45.91.116.11:4117 , piece_id=JQKTLMVS53ZOYDFUYCNFOGR6I34HVMDDMIXEMNUUNQGLK7DDPWYA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.501428606 (50m26.541001726s) [noise=true , address=45.91.116.5:4297 , piece_id=LQOF67PVBVTVBZQ5QRHNBF5WVNNQBMTZ4AQV4W3L356ZI5CJHXYA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.505258865 (50m26.544831975s) [noise=true , address=45.91.116.12:4225 , piece_id=4ZCQ536QONPUNNHTRARFZ43T7NKUBRMMX6XRJ3GGEG7RXTHCXCUA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.512713841 (50m26.552286961s) [noise=true , address=45.91.116.13:4153 , piece_id=2OIOXIU2MGF32CB47QEJMAN7EXK2DA67BJG5RXG2MYDBQZ4HTBSQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.519686393 (50m26.559259513s) [noise=true , address=45.91.116.14:4190 , piece_id=EAJ2ZHXYPAHMPWYJVVCAX22DRXADPGJBZK2Z4BL2C7CNOEKMD3BA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.521989810 (50m26.56156293s) [noise=true , address=45.91.116.10:4082 , piece_id=XVHFGF3W2OBUEKMR72XQDWUZ6WO6LKBZHS2LVTXE3B2A2DGYQKRA, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.522815411 (50m26.562388531s) [noise=true , address=45.91.116.10:4081 , piece_id=SSOGHSJBNFJXEO5F5GFDRXZOLMVOZ76UBFHPHMO4VE4TGFMW5IOQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.542621459 (50m26.58219458s) [noise=true , address=45.91.116.6:4973 , piece_id=GWRWUX6QPGIWNCQRPIITG7ST5SW4CF3ABSJ2SOH65XZKROHQU35Q, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.545162883 (50m26.584736003s) [noise=true , address=45.91.116.13:4154 , piece_id=62WP2G7VG6JN4JQFLX5DNXTNU2NZDLOZHPSXKKI6YRTQTOMX5PCQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.610468787 (50m26.650041907s) [noise=true , address=45.91.116.12:4226 , piece_id=XGA2YVX57U2ITAJGKDBVVGWS3XAT3BMOPMSGLFKRVRLCMRCZNX6A, seg_pos={0 0}, upload=724]: Done uploading piece. err: <nil> Sep 6 14:18:50.610494627 (50m26.650067737s) [seg_pos={0 0}, upload=724]: Segment reached optimal threshold of 13 pieces. Sep 6 14:18:50.610700922 (50m26.650274042s) [noise=true , address=77.90.31.13:4627 , piece_id=YJZBHPHT2NJ7LN4W4N3UTC3VZM6CQWSSD4HXDBCAEGZCJZEXNBPA, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VawvPHa7gM65pn3kxPmaMjmXqMNrutEKXTmbDxdeNP9WCgLSB): protocol: write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection; write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60908->77.90.31.13:4627: use of closed network connection Sep 6 14:18:50.610712313 (50m26.650285433s) [noise=true , address=45.91.116.14:4189 , piece_id=RZHOZRM3BBEBTLISYBZRFU7Y3ZDVO2KKEBNWI7HFYRO5N2T5DVIQ, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection; write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36570->45.91.116.14:4189: use of closed network connection Sep 6 14:18:50.610718085 (50m26.650291195s) [noise=true , address=77.90.31.18:4806 , piece_id=BHEMK7FEB2RI33OHORIXKOM3CU7BJVCKIA4D4SLKP3AEL5SDX4MA, seg_pos={0 0}, upload=724]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF Sep 6 14:18:50.610735127 (50m26.650308247s) [seg_pos={0 0}, upload=724]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:18:50.610802797 (50m26.650375917s) [upload=724]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:18:50.610807045 (50m26.650380165s) [upload=724]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:18:50.912618780 (50m26.9521919s) [upload=724]: Done uploading segments... Sep 6 14:18:50.912637626 (50m26.952210746s) [upload=724]: Upload finished. err: <nil> Duration: 30.002057743s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 24 out of 60 is failed (threshold=3) [....XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXXX] duration: 30.001916592s FAIL (2025-09-06 14:18:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:18:17.824840268 (49m53.864413388s) [upload=714]: Starting upload Sep 6 14:18:17.824891276 (49m53.864464396s) [upload=714]: Done starting upload Sep 6 14:18:17.824970337 (49m53.864543457s) [upload=714]: Uploading segments... Sep 6 14:18:17.825060700 (49m53.86463382s) [upload=714]: Got next segment. Inline: false Sep 6 14:18:17.825085918 (49m53.864659038s) [upload=714]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:18:17.825089886 (49m53.864663006s) [upload=714]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:18:18.101123703 (49m54.140696823s) [seg_pos={0 0}, upload=714]: Begin upload segment... Sep 6 14:18:18.101170393 (49m54.140743513s) [seg_pos={0 0}, upload=714]: Done begin upload segment. Sep 6 14:18:18.101192826 (49m54.140765946s) [noise=true , address=77.90.31.17:4770 , piece_id=RSEIUL2GQW7EDPM2ODZDKSXIZ6YXF7WUINTYQMVJRZ4XZAPWO53A, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.101252850 (49m54.140825971s) [noise=true , address=45.91.116.11:4119 , piece_id=VNBOSWL4H2UIFA3D6DMQQEHYZBHE4W7BCVQFERXJDLWTR3JQCD2Q, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.101548687 (49m54.141121797s) [noise=true , address=77.90.31.18:4806 , piece_id=AV7ITQJSGMYLPUUKSJMGSZZUL6GVAPGN34XPBEHJXYDWVH6Z3BZA, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.101696680 (49m54.14126979s) [noise=true , address=45.91.116.6:4974 , piece_id=TEQP67FFNLPSP7UZMRHRHFBW5FP6BPWW2YFT5LSDFQNIBSSJ65CQ, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.101845856 (49m54.141418976s) [noise=true , address=45.91.116.12:4225 , piece_id=KJVZMCGY6BI2T27BFNBLMFSRPAV2OVA6LL5SP57VKPIVWH53GNAQ, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102061618 (49m54.141634739s) [noise=true , address=77.90.31.15:4697 , piece_id=GI3WV2UWIZIDYKPL47I34YUDVTGOKCLTAM3G2WLTN5UCRUP2RE5A, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102125941 (49m54.141699061s) [noise=true , address=45.91.116.5:4298 , piece_id=2MILU4TPNK3UKRUMYA6HCTGTOV5OTJKSCLH2J54HZD4I3VJKH6GA, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102358185 (49m54.141931305s) [noise=true , address=77.90.31.13:4626 , piece_id=VP2ZW4JR32SK6K6XOZZHF736E4MSMRQ7TN7XKGQ5R7RSPDR3DN7A, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102380869 (49m54.141953979s) [noise=true , address=45.91.116.10:4083 , piece_id=RP3FAG5YIVX4X2OEEVN7FYWYYLWPKRNFMKGHSU5ZYKZYT7OOON5A, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102615057 (49m54.142188177s) [noise=true , address=77.90.31.14:4663 , piece_id=S75JDBMKWNOGNDZYQUCLE5SLLNMWORQUJT6AJISTRBR46RXMCANQ, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102624345 (49m54.142197465s) [noise=true , address=45.91.116.13:4153 , piece_id=GP2ZBVPCOYFRYSNUANNRIZ4MEN54DYRCPLH4CNY7AJDLC2CZHGJA, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102871748 (49m54.142444868s) [noise=true , address=45.91.116.10:4082 , piece_id=U4RGJKU2KPTXHFNADVS4JI5GX32HWFYBKY2MMRQ2ZIPBXG6G5K5Q, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.102878601 (49m54.142451721s) [noise=true , address=77.90.31.15:4698 , piece_id=AGM4C5SQGJ5GH4SDGMVQJRCJP7DJFUQWBBDWYVCFM4DZRGY2UZZQ, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.103134802 (49m54.142707922s) [noise=true , address=45.91.116.14:4190 , piece_id=B6DOLAETJMDMHD67VJKDYQJVXQMKCUANQWY4WQUHI2EPPNZQRYDA, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.103367346 (49m54.142940466s) [noise=true , address=45.91.116.8:4048 , piece_id=A3SFWPN3OB35ZCNT2MH7B4JIJ4GRF7OB75C5GL5ONJABZP4MBS6A, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:18.104794428 (49m54.144367558s) [noise=true , address=77.90.31.21:4879 , piece_id=5FPBFO27YMROGQQNTJYOFJTFZEH56ZMVM7MOLYYMWF2HGMNILENA, seg_pos={0 0}, upload=714]: Uploading piece... Sep 6 14:18:19.976611790 (49m56.01618491s) [upload=714]: Next returned nil segment Sep 6 14:18:19.976629604 (49m56.016202724s) [upload=714]: Waiting for error group managing segments... Sep 6 14:18:20.361087364 (49m56.400660474s) [noise=true , address=45.91.116.12:4225 , piece_id=KJVZMCGY6BI2T27BFNBLMFSRPAV2OVA6LL5SP57VKPIVWH53GNAQ, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.363201059 (49m56.402774179s) [noise=true , address=77.90.31.13:4626 , piece_id=VP2ZW4JR32SK6K6XOZZHF736E4MSMRQ7TN7XKGQ5R7RSPDR3DN7A, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.367857980 (49m56.40743109s) [noise=true , address=77.90.31.15:4697 , piece_id=GI3WV2UWIZIDYKPL47I34YUDVTGOKCLTAM3G2WLTN5UCRUP2RE5A, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.380788011 (49m56.420361121s) [noise=true , address=45.91.116.5:4298 , piece_id=2MILU4TPNK3UKRUMYA6HCTGTOV5OTJKSCLH2J54HZD4I3VJKH6GA, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.397617245 (49m56.437190365s) [noise=true , address=77.90.31.21:4879 , piece_id=5FPBFO27YMROGQQNTJYOFJTFZEH56ZMVM7MOLYYMWF2HGMNILENA, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.401678046 (49m56.441251166s) [noise=true , address=45.91.116.6:4974 , piece_id=TEQP67FFNLPSP7UZMRHRHFBW5FP6BPWW2YFT5LSDFQNIBSSJ65CQ, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.406389791 (49m56.445962901s) [noise=true , address=77.90.31.14:4663 , piece_id=S75JDBMKWNOGNDZYQUCLE5SLLNMWORQUJT6AJISTRBR46RXMCANQ, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.409159661 (49m56.448732781s) [noise=true , address=45.91.116.13:4153 , piece_id=GP2ZBVPCOYFRYSNUANNRIZ4MEN54DYRCPLH4CNY7AJDLC2CZHGJA, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.411373387 (49m56.450946507s) [noise=true , address=77.90.31.15:4698 , piece_id=AGM4C5SQGJ5GH4SDGMVQJRCJP7DJFUQWBBDWYVCFM4DZRGY2UZZQ, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.434403556 (49m56.473976676s) [noise=true , address=45.91.116.11:4119 , piece_id=VNBOSWL4H2UIFA3D6DMQQEHYZBHE4W7BCVQFERXJDLWTR3JQCD2Q, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.445279357 (49m56.484852477s) [noise=true , address=77.90.31.17:4770 , piece_id=RSEIUL2GQW7EDPM2ODZDKSXIZ6YXF7WUINTYQMVJRZ4XZAPWO53A, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.445549294 (49m56.485122414s) [noise=true , address=45.91.116.8:4048 , piece_id=A3SFWPN3OB35ZCNT2MH7B4JIJ4GRF7OB75C5GL5ONJABZP4MBS6A, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.510547680 (49m56.55012079s) [noise=true , address=45.91.116.14:4190 , piece_id=B6DOLAETJMDMHD67VJKDYQJVXQMKCUANQWY4WQUHI2EPPNZQRYDA, seg_pos={0 0}, upload=714]: Done uploading piece. err: <nil> Sep 6 14:18:20.510575784 (49m56.550148904s) [seg_pos={0 0}, upload=714]: Segment reached optimal threshold of 13 pieces. Sep 6 14:18:20.510754816 (49m56.550327936s) [noise=true , address=45.91.116.10:4082 , piece_id=U4RGJKU2KPTXHFNADVS4JI5GX32HWFYBKY2MMRQ2ZIPBXG6G5K5Q, seg_pos={0 0}, upload=714]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: expected piece hash; context canceled; EOF Sep 6 14:18:20.510806595 (49m56.550379715s) [noise=true , address=77.90.31.18:4806 , piece_id=AV7ITQJSGMYLPUUKSJMGSZZUL6GVAPGN34XPBEHJXYDWVH6Z3BZA, seg_pos={0 0}, upload=714]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF Sep 6 14:18:20.510889784 (49m56.550462905s) [noise=true , address=45.91.116.10:4083 , piece_id=RP3FAG5YIVX4X2OEEVN7FYWYYLWPKRNFMKGHSU5ZYKZYT7OOON5A, seg_pos={0 0}, upload=714]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:55210->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:55210->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55210->45.91.116.10:4083: use of closed network connection Sep 6 14:18:20.510898532 (49m56.550471652s) [seg_pos={0 0}, upload=714]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:18:20.510960270 (49m56.55053339s) [upload=714]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:18:20.510964137 (49m56.550537257s) [upload=714]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:18:20.883021228 (49m56.922594349s) [upload=714]: Done uploading segments... Sep 6 14:18:20.883039062 (49m56.922612172s) [upload=714]: Upload finished. err: <nil> Duration: 30.001940648s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 23 out of 60 is failed (threshold=3) [.....XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XXX] duration: 30.001796334s FAIL (2025-09-06 14:18:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:17:34.635046442 (49m10.674619562s) [upload=703]: Starting upload Sep 6 14:17:34.635103621 (49m10.674676741s) [upload=703]: Done starting upload Sep 6 14:17:34.635189044 (49m10.674762165s) [upload=703]: Uploading segments... Sep 6 14:17:34.635362657 (49m10.674935797s) [upload=703]: Got next segment. Inline: false Sep 6 14:17:34.635412964 (49m10.674986084s) [upload=703]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:17:34.635419336 (49m10.674992456s) [upload=703]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:17:34.884663711 (49m10.924236831s) [seg_pos={0 0}, upload=703]: Begin upload segment... Sep 6 14:17:34.884712745 (49m10.924285865s) [seg_pos={0 0}, upload=703]: Done begin upload segment. Sep 6 14:17:34.884741931 (49m10.924315051s) [noise=true , address=45.91.116.14:4190 , piece_id=N6XBE6FSRGXG4VU6A7ITUJZXVFEQ4J6WQXJCINJJJPCGE2Y5ZGGA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.884783620 (49m10.92435674s) [noise=true , address=45.91.116.14:4191 , piece_id=ERR7JQZ3EELYE76T2DBDXWGWY7EDESNDEUE45W7V5JTWP7CMVGFQ, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.885146716 (49m10.924719836s) [noise=true , address=45.91.116.10:4081 , piece_id=K7HWNIQ45KZDO5C4V7BLACU2O7E5EB7XZTEGT53EITVWSUXD3OZA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.885331530 (49m10.92490465s) [noise=true , address=45.91.116.9:4263 , piece_id=IOWHTTRSTAECKCE6JD5JGJBRUQ6IDLYLAS6PAG232MUQ6RXT7LLA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.885487818 (49m10.925060938s) [noise=true , address=45.91.116.13:4154 , piece_id=J7NTCZ4VRZVLUUPVPPGNZQQN4DJG4FSBK7TGUIRPUOGNKDCDXXKA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.885672361 (49m10.925245481s) [noise=true , address=45.91.116.10:4082 , piece_id=7HW6ADIUSDO4MR4EJJ4AI7OEO6MIHHLLI336VTFASRW7HTYYC6UQ, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.885787732 (49m10.925360852s) [noise=true , address=77.90.31.15:4699 , piece_id=GI6KJES4YCVNLL3AYXX7542IM543WXOOGDIY4W6C6ZEWN6SIJ66A, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.886052258 (49m10.925625378s) [noise=true , address=77.90.31.12:4589 , piece_id=KLUOUILBHRIQCVQ5QMUO3N64Y3ZMQRSYQM555YKTABOS36HAKXYA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.886321624 (49m10.925894744s) [noise=true , address=45.91.116.14:4189 , piece_id=3GNZJXZEDSDCAQU7NSDU3XQKTGSFO77NLBXPJ25WYYUDXPOCTZDA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.886569178 (49m10.926142288s) [noise=true , address=45.91.116.5:4297 , piece_id=WXHYD7XLYUKNX7ICFRT2Y4EYQU5REM4GS367QBO4GE3HFXAHUY2A, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.887209042 (49m10.926782162s) [noise=true , address=45.91.116.6:4975 , piece_id=FSTUZHUWOTPXIIN6F7V3FXZ52I4D6Z47LKYTFBW2L2L26QJIATWA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.887670284 (49m10.927243404s) [noise=true , address=45.91.116.5:4298 , piece_id=3HGHZB63MHCRONDHIIFFI6JQCYHGK5U3XRJWJNFWYTEP3JXD7D7Q, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.888497186 (49m10.928070306s) [noise=true , address=45.91.116.11:4119 , piece_id=WH3GXLL6NQO4J6GS4PYMHTTEBD52L7XH5FJDWDBI542UYAG6V5PA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.889506297 (49m10.929079417s) [noise=true , address=45.91.116.8:4048 , piece_id=7AY2QLBAD65WHKA6BJTINY3JFROH7R4PVO3V6RT2EDBHWU3JSUFA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.890282233 (49m10.929855353s) [noise=true , address=45.91.116.13:4153 , piece_id=ZAX73LN5CVY7X7PJXUVJ6LJVNCTJVUQSFUS6ZKWCMULXYYXZIKYA, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:34.890801887 (49m10.930375007s) [noise=true , address=45.91.116.12:4227 , piece_id=G2USF6C4EINXTGWUON62UTO5CZDACQVCSZB4F767UDCMBCKIQY7A, seg_pos={0 0}, upload=703]: Uploading piece... Sep 6 14:17:36.647894360 (49m12.68746748s) [upload=703]: Next returned nil segment Sep 6 14:17:36.647908827 (49m12.687481947s) [upload=703]: Waiting for error group managing segments... Sep 6 14:17:37.073893320 (49m13.11346644s) [noise=true , address=45.91.116.12:4227 , piece_id=G2USF6C4EINXTGWUON62UTO5CZDACQVCSZB4F767UDCMBCKIQY7A, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.088975440 (49m13.128555663s) [noise=true , address=45.91.116.14:4190 , piece_id=N6XBE6FSRGXG4VU6A7ITUJZXVFEQ4J6WQXJCINJJJPCGE2Y5ZGGA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.091188725 (49m13.130761845s) [noise=true , address=45.91.116.10:4082 , piece_id=7HW6ADIUSDO4MR4EJJ4AI7OEO6MIHHLLI336VTFASRW7HTYYC6UQ, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.098348415 (49m13.137921525s) [noise=true , address=45.91.116.8:4048 , piece_id=7AY2QLBAD65WHKA6BJTINY3JFROH7R4PVO3V6RT2EDBHWU3JSUFA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.101466782 (49m13.141039902s) [noise=true , address=45.91.116.6:4975 , piece_id=FSTUZHUWOTPXIIN6F7V3FXZ52I4D6Z47LKYTFBW2L2L26QJIATWA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.117394100 (49m13.15696721s) [noise=true , address=45.91.116.5:4297 , piece_id=WXHYD7XLYUKNX7ICFRT2Y4EYQU5REM4GS367QBO4GE3HFXAHUY2A, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.121712323 (49m13.161285443s) [noise=true , address=45.91.116.11:4119 , piece_id=WH3GXLL6NQO4J6GS4PYMHTTEBD52L7XH5FJDWDBI542UYAG6V5PA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.126786853 (49m13.166359973s) [noise=true , address=45.91.116.10:4081 , piece_id=K7HWNIQ45KZDO5C4V7BLACU2O7E5EB7XZTEGT53EITVWSUXD3OZA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.140093436 (49m13.179666556s) [noise=true , address=45.91.116.13:4153 , piece_id=ZAX73LN5CVY7X7PJXUVJ6LJVNCTJVUQSFUS6ZKWCMULXYYXZIKYA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.141110983 (49m13.180684103s) [noise=true , address=77.90.31.12:4589 , piece_id=KLUOUILBHRIQCVQ5QMUO3N64Y3ZMQRSYQM555YKTABOS36HAKXYA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.152459949 (49m13.192033069s) [noise=true , address=45.91.116.14:4189 , piece_id=3GNZJXZEDSDCAQU7NSDU3XQKTGSFO77NLBXPJ25WYYUDXPOCTZDA, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.206966369 (49m13.246539489s) [noise=true , address=45.91.116.14:4191 , piece_id=ERR7JQZ3EELYE76T2DBDXWGWY7EDESNDEUE45W7V5JTWP7CMVGFQ, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.303171871 (49m13.342744981s) [noise=true , address=45.91.116.5:4298 , piece_id=3HGHZB63MHCRONDHIIFFI6JQCYHGK5U3XRJWJNFWYTEP3JXD7D7Q, seg_pos={0 0}, upload=703]: Done uploading piece. err: <nil> Sep 6 14:17:37.303195917 (49m13.342769027s) [seg_pos={0 0}, upload=703]: Segment reached optimal threshold of 13 pieces. Sep 6 14:17:37.303565704 (49m13.343138824s) [noise=true , address=45.91.116.13:4154 , piece_id=J7NTCZ4VRZVLUUPVPPGNZQQN4DJG4FSBK7TGUIRPUOGNKDCDXXKA, seg_pos={0 0}, upload=703]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1vnK2UMEryJTrW3L6KckWmgWmzueeBkbDqG8AoRBQWHELBruRN): protocol: write tcp 188.166.225.170:35952->45.91.116.13:4154: use of closed network connection; write tcp 188.166.225.170:35952->45.91.116.13:4154: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35952->45.91.116.13:4154: use of closed network connection Sep 6 14:17:37.303585683 (49m13.343158803s) [noise=true , address=45.91.116.9:4263 , piece_id=IOWHTTRSTAECKCE6JD5JGJBRUQ6IDLYLAS6PAG232MUQ6RXT7LLA, seg_pos={0 0}, upload=703]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF): protocol: write tcp 188.166.225.170:55250->45.91.116.9:4263: use of closed network connection; write tcp 188.166.225.170:55250->45.91.116.9:4263: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:55250->45.91.116.9:4263: use of closed network connection Sep 6 14:17:37.303616491 (49m13.343189601s) [noise=true , address=77.90.31.15:4699 , piece_id=GI6KJES4YCVNLL3AYXX7542IM543WXOOGDIY4W6C6ZEWN6SIJ66A, seg_pos={0 0}, upload=703]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: write tcp 188.166.225.170:54716->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:54716->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54716->77.90.31.15:4699: use of closed network connection Sep 6 14:17:37.303632742 (49m13.343205862s) [seg_pos={0 0}, upload=703]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:17:37.303717876 (49m13.343290996s) [upload=703]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:17:37.303723616 (49m13.343296737s) [upload=703]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:17:37.618118699 (49m13.657691819s) [upload=703]: Done uploading segments... Sep 6 14:17:37.618132416 (49m13.657705536s) [upload=703]: Upload finished. err: <nil> Duration: 30.001807785s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [......XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.XX] duration: 30.002315307s FAIL (2025-09-06 14:17:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:17:04.380265425 (48m40.419838545s) [upload=695]: Starting upload Sep 6 14:17:04.380345939 (48m40.419919049s) [upload=695]: Done starting upload Sep 6 14:17:04.380422576 (48m40.419995696s) [upload=695]: Uploading segments... Sep 6 14:17:04.380654720 (48m40.42022784s) [upload=695]: Got next segment. Inline: false Sep 6 14:17:04.380681662 (48m40.420254782s) [upload=695]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:17:04.380686130 (48m40.42025925s) [upload=695]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:17:04.874019477 (48m40.913592587s) [seg_pos={0 0}, upload=695]: Begin upload segment... Sep 6 14:17:04.874065484 (48m40.913638594s) [seg_pos={0 0}, upload=695]: Done begin upload segment. Sep 6 14:17:04.874086595 (48m40.913659715s) [noise=true , address=77.90.31.12:4592 , piece_id=GWF55IEWUCDTK4EI3Y7EUYC5YUDDOG2SARA7SJWNEABSQMDYINIQ, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.874098739 (48m40.913671869s) [noise=true , address=45.91.116.4:4940 , piece_id=SRI6THJOGQPO2MZ3RHRYIPJQRNVLSIIORSG2JE4YCVJLMWZNWZBQ, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.874452345 (48m40.914025465s) [noise=true , address=45.91.116.10:4083 , piece_id=36O3ZBX5VMSM6TNQCUAJ55C66F55KQHGQNWWPTEXMWNPNRCDPO3Q, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.874457455 (48m40.914030575s) [noise=true , address=45.91.116.10:4082 , piece_id=WODLMMHEF4RUS5MHWTBZK4CLQ4GGIV3HE22E4V3QIM6GSQ337QWQ, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.874770624 (48m40.914343744s) [noise=true , address=77.90.31.10:4518 , piece_id=WMOCOGIZA6GUR6HWICM6FOKMOM2V2Z5CXCTG4QQSARYIJ5UAHDBQ, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.875048966 (48m40.914622076s) [noise=true , address=45.91.116.6:4975 , piece_id=A3DYKJT2J5SDOQ564YF4ZM52VLTUMCFAWPWMFEAC7UDLA7UQXEPQ, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.875301049 (48m40.914874169s) [noise=true , address=45.91.116.13:4154 , piece_id=DVYM3HNWVVT3POXVMHS32TFTSOKLD76ILO3ETNXCSLAMD2NIJL4Q, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.875582027 (48m40.915155147s) [noise=true , address=77.90.31.16:4734 , piece_id=IACUYX3QWX62R4IR6M7BEKJNCOU3R4XAYXF4WPTIJUTD3WX6LT6A, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.875802909 (48m40.915376029s) [noise=true , address=77.90.31.18:4807 , piece_id=INOKAT3ZP6GW6BTHHGUD2YEQTJKGVUKLUJ32MQU6DAPFJSWJBD2Q, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.876019163 (48m40.915592283s) [noise=true , address=77.90.31.14:4661 , piece_id=PBKLGTQF7DLUHQELGBLI4Y2W6QFBU7WHE3LUDIH4GERZ4LB4YOIA, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.876239374 (48m40.915812495s) [noise=true , address=45.91.116.8:4045 , piece_id=ZUG32JLSJA3QCPQO7VHKLR4YJGT3QPOSNF5BOTUA2ITZN2PWETZA, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.876432334 (48m40.916005454s) [noise=true , address=45.91.116.6:4974 , piece_id=LBO7MAFI5I76U76L7EMCSMA7MVHL7XIV6VJR6SUFL3HC6A3675WQ, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.876623620 (48m40.91619674s) [noise=true , address=45.91.116.12:4227 , piece_id=K7Q2VE3O5YLYSAZ45LW6RTXT3GGPQDXNIJNN5LVWJSL2CWGBCZRA, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.876815477 (48m40.916388597s) [noise=true , address=45.91.116.12:4226 , piece_id=DYWAX4ZOSQZB6INSWZXKTZ7WMW7UTNPDCMSH22SBNYPLPNLQQU7Q, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.877032873 (48m40.916605983s) [noise=true , address=77.90.31.18:4806 , piece_id=VTX5QLPYSTMCAO36MQYXPZD3QQJMB42CZPM5A2GBBNIPSKSYYFIA, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:04.878168276 (48m40.917741386s) [noise=true , address=45.91.116.8:4048 , piece_id=42DORTGAZF6YMFRYBXJKAJRQHVWOATMEIDDDNL2RFEP6RHME472Q, seg_pos={0 0}, upload=695]: Uploading piece... Sep 6 14:17:06.668088425 (48m42.707661535s) [upload=695]: Next returned nil segment Sep 6 14:17:06.668105709 (48m42.707678829s) [upload=695]: Waiting for error group managing segments... Sep 6 14:17:07.027149859 (48m43.066722979s) [noise=true , address=77.90.31.12:4592 , piece_id=GWF55IEWUCDTK4EI3Y7EUYC5YUDDOG2SARA7SJWNEABSQMDYINIQ, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.030154739 (48m43.069727859s) [noise=true , address=45.91.116.6:4974 , piece_id=LBO7MAFI5I76U76L7EMCSMA7MVHL7XIV6VJR6SUFL3HC6A3675WQ, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.039697298 (48m43.079270429s) [noise=true , address=45.91.116.10:4083 , piece_id=36O3ZBX5VMSM6TNQCUAJ55C66F55KQHGQNWWPTEXMWNPNRCDPO3Q, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.096821307 (48m43.136394427s) [noise=true , address=45.91.116.12:4226 , piece_id=DYWAX4ZOSQZB6INSWZXKTZ7WMW7UTNPDCMSH22SBNYPLPNLQQU7Q, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.097336103 (48m43.136909223s) [noise=true , address=45.91.116.10:4082 , piece_id=WODLMMHEF4RUS5MHWTBZK4CLQ4GGIV3HE22E4V3QIM6GSQ337QWQ, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.115184937 (48m43.154758107s) [noise=true , address=77.90.31.18:4806 , piece_id=VTX5QLPYSTMCAO36MQYXPZD3QQJMB42CZPM5A2GBBNIPSKSYYFIA, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.121919904 (48m43.161493024s) [noise=true , address=45.91.116.12:4227 , piece_id=K7Q2VE3O5YLYSAZ45LW6RTXT3GGPQDXNIJNN5LVWJSL2CWGBCZRA, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.124797591 (48m43.164370711s) [noise=true , address=45.91.116.13:4154 , piece_id=DVYM3HNWVVT3POXVMHS32TFTSOKLD76ILO3ETNXCSLAMD2NIJL4Q, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.157851700 (48m43.19742482s) [noise=true , address=77.90.31.10:4518 , piece_id=WMOCOGIZA6GUR6HWICM6FOKMOM2V2Z5CXCTG4QQSARYIJ5UAHDBQ, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.202144856 (48m43.241717976s) [noise=true , address=77.90.31.16:4734 , piece_id=IACUYX3QWX62R4IR6M7BEKJNCOU3R4XAYXF4WPTIJUTD3WX6LT6A, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.228208480 (48m43.2677816s) [noise=true , address=77.90.31.18:4807 , piece_id=INOKAT3ZP6GW6BTHHGUD2YEQTJKGVUKLUJ32MQU6DAPFJSWJBD2Q, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.389279234 (48m43.428852354s) [noise=true , address=45.91.116.8:4048 , piece_id=42DORTGAZF6YMFRYBXJKAJRQHVWOATMEIDDDNL2RFEP6RHME472Q, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.877017043 (48m43.916590163s) [noise=true , address=45.91.116.8:4045 , piece_id=ZUG32JLSJA3QCPQO7VHKLR4YJGT3QPOSNF5BOTUA2ITZN2PWETZA, seg_pos={0 0}, upload=695]: Done uploading piece. err: <nil> Sep 6 14:17:07.877052641 (48m43.916625761s) [seg_pos={0 0}, upload=695]: Segment reached optimal threshold of 13 pieces. Sep 6 14:17:07.877166779 (48m43.916739899s) [noise=true , address=77.90.31.14:4661 , piece_id=PBKLGTQF7DLUHQELGBLI4Y2W6QFBU7WHE3LUDIH4GERZ4LB4YOIA, seg_pos={0 0}, upload=695]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 6 14:17:07.877239398 (48m43.916812518s) [noise=true , address=45.91.116.6:4975 , piece_id=A3DYKJT2J5SDOQ564YF4ZM52VLTUMCFAWPWMFEAC7UDLA7UQXEPQ, seg_pos={0 0}, upload=695]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Sep 6 14:17:07.877321646 (48m43.916894766s) [noise=true , address=45.91.116.4:4940 , piece_id=SRI6THJOGQPO2MZ3RHRYIPJQRNVLSIIORSG2JE4YCVJLMWZNWZBQ, seg_pos={0 0}, upload=695]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ifieiAT4XHJHsEsrwSqqnRmctV79VUkoWwq3aCQjxWqK8MiCe): protocol: expected piece hash; context canceled; EOF Sep 6 14:17:07.877360259 (48m43.916933379s) [seg_pos={0 0}, upload=695]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:17:07.877440573 (48m43.917013693s) [upload=695]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:17:07.877444600 (48m43.91701772s) [upload=695]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:17:08.183584908 (48m44.223158029s) [upload=695]: Done uploading segments... Sep 6 14:17:08.183603354 (48m44.223176474s) [upload=695]: Upload finished. err: <nil> Duration: 30.002329233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [X......XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.X] duration: 30.001882598s FAIL (2025-09-06 14:17:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:16:34.142003510 (48m10.18157663s) [upload=690]: Starting upload Sep 6 14:16:34.142066670 (48m10.18163979s) [upload=690]: Done starting upload Sep 6 14:16:34.142174276 (48m10.181747396s) [upload=690]: Uploading segments... Sep 6 14:16:34.142303103 (48m10.181876223s) [upload=690]: Got next segment. Inline: false Sep 6 14:16:34.142352588 (48m10.181925708s) [upload=690]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:16:34.142357327 (48m10.181930447s) [upload=690]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:16:34.678108220 (48m10.71768134s) [seg_pos={0 0}, upload=690]: Begin upload segment... Sep 6 14:16:34.678160329 (48m10.717733449s) [seg_pos={0 0}, upload=690]: Done begin upload segment. Sep 6 14:16:34.678197821 (48m10.717770941s) [noise=true , address=77.90.31.15:4697 , piece_id=O6GI6LPCDUYU44FE73KDKCNJS4FT5LQFE22C4UP4IGWL22YJA43Q, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.678218631 (48m10.717791761s) [noise=true , address=77.90.31.13:4628 , piece_id=BSBQKQYXB2NLDYG67BSBBGWFV3ZQQHA5S7IW45CW5OLLYWIHVXAQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.678701705 (48m10.718274825s) [noise=true , address=45.91.116.10:4082 , piece_id=K27E7KHU3WIEBJGS4DT7ZH5STZ4HIKTAS7WFGZQ5BBAZKE2TMVMQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.680071927 (48m10.719645037s) [noise=true , address=77.90.31.10:4518 , piece_id=ZNQMRVOMDY2GYQPTYO6FMFQ54QLLCT6DPFZNIDC6DRP2ZSK445HA, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.680159726 (48m10.719732856s) [noise=true , address=45.91.116.9:4262 , piece_id=Y2RH2SA4M3ZGWVOS24LKHKVRRKPXXT57OT77FZ2N3EZGL537JVSQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.680418460 (48m10.71999158s) [noise=true , address=45.91.116.13:4153 , piece_id=SHFHDAL22MEIJ4RIKZFVNOQIZPEGUVCHXLJM6R7KQKJ4LRHUFX5A, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.680634504 (48m10.720207624s) [noise=true , address=77.90.31.21:4879 , piece_id=RA5WNYPHUHPJ5R3C3DSSZRFJC3TVNJSNGX3VNXFFPGYN7ISK7HOQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.680687165 (48m10.720260285s) [noise=true , address=45.91.116.6:4974 , piece_id=3PEZCLVF3RD4IIJ6LPQZ5YTYSVA3IHOQ53EYC2CZ2VADWVMZOZCQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.680981318 (48m10.720554428s) [noise=true , address=45.91.116.7:4010 , piece_id=XRD24HM3NYNOJQTUQUKN62372R7JTQKSAGCTESPFZPEMQCGYOOOQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.681060549 (48m10.72063367s) [noise=true , address=45.91.116.14:4190 , piece_id=QTMJ63AVPFTT7V44U7WTBRYVAQVHSZKXN2PGVMOAWCHAZTXK2TLQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.681329123 (48m10.720902233s) [noise=true , address=77.90.31.19:4843 , piece_id=SYONDCVUQSR4WA3DLE4OSZFX4MUAH5C3KEX6CA6A4YHTCYTCGL4Q, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.681619328 (48m10.721192448s) [noise=true , address=77.90.31.14:4662 , piece_id=F2ZY2Y4TJ27E7NOTXTKCHFEMNXDODVTJWZ5HW6V5CLSEYHWE5M5Q, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.682023792 (48m10.721596912s) [noise=true , address=45.91.116.12:4225 , piece_id=7W5QZ544KAGGDUH23HNLWNX7ONSPKN6J6OV2JTFDQXHHZZ4MNDKQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.682300392 (48m10.721873512s) [noise=true , address=45.91.116.5:4298 , piece_id=PABPOE3KJFOZBK5ZQCV6COBQ6PLEIMQZB77UP75CFWDSNYHT3M5Q, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.682575027 (48m10.722148147s) [noise=true , address=45.91.116.12:4227 , piece_id=UMTKEMLFY7RGPHIG3SIHXR3QIROW56XNNSVUAYVKRWGZC7FQIHJQ, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:34.682866165 (48m10.722439285s) [noise=true , address=45.91.116.11:4119 , piece_id=CDBNLDO7XL5CNFXVZBORQYZH4FHI4VVCINDTERVNKOEV2VJUUQDA, seg_pos={0 0}, upload=690]: Uploading piece... Sep 6 14:16:36.557160306 (48m12.596733416s) [upload=690]: Next returned nil segment Sep 6 14:16:36.557173872 (48m12.596746992s) [upload=690]: Waiting for error group managing segments... Sep 6 14:16:36.866474136 (48m12.906047256s) [noise=true , address=45.91.116.6:4974 , piece_id=3PEZCLVF3RD4IIJ6LPQZ5YTYSVA3IHOQ53EYC2CZ2VADWVMZOZCQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.867309224 (48m12.906882344s) [noise=true , address=45.91.116.12:4225 , piece_id=7W5QZ544KAGGDUH23HNLWNX7ONSPKN6J6OV2JTFDQXHHZZ4MNDKQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.879846946 (48m12.919420066s) [noise=true , address=45.91.116.14:4190 , piece_id=QTMJ63AVPFTT7V44U7WTBRYVAQVHSZKXN2PGVMOAWCHAZTXK2TLQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.883309751 (48m12.922882871s) [noise=true , address=45.91.116.11:4119 , piece_id=CDBNLDO7XL5CNFXVZBORQYZH4FHI4VVCINDTERVNKOEV2VJUUQDA, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.884144288 (48m12.923717408s) [noise=true , address=45.91.116.5:4298 , piece_id=PABPOE3KJFOZBK5ZQCV6COBQ6PLEIMQZB77UP75CFWDSNYHT3M5Q, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.893863216 (48m12.933436336s) [noise=true , address=45.91.116.13:4153 , piece_id=SHFHDAL22MEIJ4RIKZFVNOQIZPEGUVCHXLJM6R7KQKJ4LRHUFX5A, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.898978955 (48m12.938552075s) [noise=true , address=77.90.31.19:4843 , piece_id=SYONDCVUQSR4WA3DLE4OSZFX4MUAH5C3KEX6CA6A4YHTCYTCGL4Q, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.935038977 (48m12.974612087s) [noise=true , address=45.91.116.7:4010 , piece_id=XRD24HM3NYNOJQTUQUKN62372R7JTQKSAGCTESPFZPEMQCGYOOOQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.949115272 (48m12.988688393s) [noise=true , address=45.91.116.12:4227 , piece_id=UMTKEMLFY7RGPHIG3SIHXR3QIROW56XNNSVUAYVKRWGZC7FQIHJQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.951665181 (48m12.991238292s) [noise=true , address=45.91.116.10:4082 , piece_id=K27E7KHU3WIEBJGS4DT7ZH5STZ4HIKTAS7WFGZQ5BBAZKE2TMVMQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.985637388 (48m13.025210498s) [noise=true , address=77.90.31.15:4697 , piece_id=O6GI6LPCDUYU44FE73KDKCNJS4FT5LQFE22C4UP4IGWL22YJA43Q, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.996671622 (48m13.036244743s) [noise=true , address=77.90.31.13:4628 , piece_id=BSBQKQYXB2NLDYG67BSBBGWFV3ZQQHA5S7IW45CW5OLLYWIHVXAQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.999582874 (48m13.039155994s) [noise=true , address=77.90.31.21:4879 , piece_id=RA5WNYPHUHPJ5R3C3DSSZRFJC3TVNJSNGX3VNXFFPGYN7ISK7HOQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: <nil> Sep 6 14:16:36.999598593 (48m13.039171773s) [seg_pos={0 0}, upload=690]: Segment reached optimal threshold of 13 pieces. Sep 6 14:16:36.999866626 (48m13.039439746s) [noise=true , address=77.90.31.14:4662 , piece_id=F2ZY2Y4TJ27E7NOTXTKCHFEMNXDODVTJWZ5HW6V5CLSEYHWE5M5Q, seg_pos={0 0}, upload=690]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n): protocol: write tcp 188.166.225.170:41052->77.90.31.14:4662: use of closed network connection; write tcp 188.166.225.170:41052->77.90.31.14:4662: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41052->77.90.31.14:4662: use of closed network connection Sep 6 14:16:36.999928074 (48m13.039501194s) [noise=true , address=45.91.116.9:4262 , piece_id=Y2RH2SA4M3ZGWVOS24LKHKVRRKPXXT57OT77FZ2N3EZGL537JVSQ, seg_pos={0 0}, upload=690]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: write tcp 188.166.225.170:51922->45.91.116.9:4262: use of closed network connection; write tcp 188.166.225.170:51922->45.91.116.9:4262: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51922->45.91.116.9:4262: use of closed network connection Sep 6 14:16:36.999939656 (48m13.039512776s) [noise=true , address=77.90.31.10:4518 , piece_id=ZNQMRVOMDY2GYQPTYO6FMFQ54QLLCT6DPFZNIDC6DRP2ZSK445HA, seg_pos={0 0}, upload=690]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3): protocol: write tcp 188.166.225.170:35352->77.90.31.10:4518: use of closed network connection; write tcp 188.166.225.170:35352->77.90.31.10:4518: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35352->77.90.31.10:4518: use of closed network connection Sep 6 14:16:36.999956388 (48m13.039529498s) [seg_pos={0 0}, upload=690]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:16:37.000026522 (48m13.039599652s) [upload=690]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:16:37.000030670 (48m13.03960379s) [upload=690]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:16:37.310977284 (48m13.350550405s) [upload=690]: Done uploading segments... Sep 6 14:16:37.310996221 (48m13.350569341s) [upload=690]: Upload finished. err: <nil> Duration: 30.001901814s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [XX......XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X.] duration: 19.48946566s FAIL (2025-09-06 14:15:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:14:13.283219758 (45m49.322792878s) [upload=659]: Starting upload Sep 6 14:14:13.283280274 (45m49.322853435s) [upload=659]: Done starting upload Sep 6 14:14:13.283345298 (45m49.322918418s) [upload=659]: Uploading segments... Sep 6 14:14:13.283587732 (45m49.323160842s) [upload=659]: Got next segment. Inline: false Sep 6 14:14:13.283640403 (45m49.323213513s) [upload=659]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:14:13.283646044 (45m49.323219164s) [upload=659]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:14:13.751750013 (45m49.791323133s) [seg_pos={0 0}, upload=659]: Begin upload segment... Sep 6 14:14:13.751788868 (45m49.791361988s) [seg_pos={0 0}, upload=659]: Done begin upload segment. Sep 6 14:14:13.751811311 (45m49.791384431s) [noise=true , address=77.90.31.10:4519 , piece_id=K65NOSR2VT7SIHF5NC6XLNIBMV65FOQ6H6747DIVGTIJ374YQ2VQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.751856537 (45m49.791429657s) [noise=true , address=45.91.116.4:4940 , piece_id=ZGP3LOKW7RIFIBJNKLCB3FXV72TMJRWBEWW7J6VWH7FMS7WE3CGQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752167342 (45m49.791740452s) [noise=true , address=77.90.31.18:4807 , piece_id=WQQJYDTZJCW3KX7DIYULZ7W2TAMCI5DMZQB6XEH7I7X2KZBBTWEA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752484548 (45m49.792057668s) [noise=true , address=45.91.116.7:4009 , piece_id=Y73WHM7ZZZTYQZ4QDGJ3WLPZB4I4BBWBARMNP4VAZPBCF7YIMZ4A, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752562708 (45m49.792135888s) [noise=true , address=45.91.116.14:4189 , piece_id=DPXX4H72DZM75BSQBPD47XK2QBDL2VMXYNHNKLN3YWIUGQOSLYYQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752754856 (45m49.792327976s) [noise=true , address=45.91.116.10:4082 , piece_id=CSWXLBIH75P7XANYYFVWZW273IV6XIVCKCYB5B4XB2BU2QFV3NVA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753037867 (45m49.792610987s) [noise=true , address=45.91.116.9:4262 , piece_id=3Z3FV7DAS3G3Z6LVVPB2CBCPRZIVCG2NZ3BHQ5DGAGCX4YLAO52A, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753254622 (45m49.792827742s) [noise=true , address=77.90.31.18:4805 , piece_id=DTF6UKXIML7VCJYZMCNUMOAIXC436BDQ3J7YOXWYSLV22SZAGEGA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753468501 (45m49.793041611s) [noise=true , address=45.91.116.8:4048 , piece_id=3QI6WCPSLVBHVGGCX2UYILYJ5JOJTZL6HPMIGTJYIAMO7CLQVOZA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753949412 (45m49.793522532s) [noise=true , address=45.91.116.4:4939 , piece_id=P7UAZR7JTF53TDOO43EILUFQYW2ZWSH4XXHQSPXZ37XMYJG34YBQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754168070 (45m49.79374118s) [noise=true , address=45.91.116.14:4191 , piece_id=C7HGF72ETJN6MJO5YA7527LIGMRXFHNT7ZJPAFWEJV5JLIMKQF7A, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754382671 (45m49.793955781s) [noise=true , address=45.91.116.12:4226 , piece_id=2NJ7KGH6WN7J5DGXLOAQLU5I7SYEV6BGMP2YP7AAM5RWGH5Q6IJA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754622750 (45m49.79419586s) [noise=true , address=77.90.31.19:4841 , piece_id=MWW64FHFPHG4L5BN42IIZYVE263OE7WEXGTCZQEDTX5VGLFYY7IA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754843513 (45m49.794416633s) [noise=true , address=77.90.31.14:4663 , piece_id=PCCJUEMOBQXABFBC465Y4HT7JDYJKYKZDS6KH5CDKBRBIFKRSNWQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.755048144 (45m49.794621264s) [noise=true , address=45.91.116.8:4047 , piece_id=NWFXEENZKBXLMVA5T2XWRBGQUUSKRSHT4QKCEQA5Q5BTQDJHUCQA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.755242225 (45m49.794815345s) [noise=true , address=45.91.116.5:4299 , piece_id=NGSDLHD5FZNMNHEZ7ZORWJ2CYGITBC653MWJNIVG7FPEZMICBR5Q, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:15.614631426 (45m51.654204546s) [upload=659]: Next returned nil segment Sep 6 14:14:15.614646394 (45m51.654219505s) [upload=659]: Waiting for error group managing segments... Sep 6 14:14:15.935223928 (45m51.974797048s) [noise=true , address=45.91.116.9:4262 , piece_id=3Z3FV7DAS3G3Z6LVVPB2CBCPRZIVCG2NZ3BHQ5DGAGCX4YLAO52A, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:15.955797296 (45m51.995370416s) [noise=true , address=77.90.31.18:4807 , piece_id=WQQJYDTZJCW3KX7DIYULZ7W2TAMCI5DMZQB6XEH7I7X2KZBBTWEA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:15.967768253 (45m52.007341373s) [noise=true , address=45.91.116.8:4048 , piece_id=3QI6WCPSLVBHVGGCX2UYILYJ5JOJTZL6HPMIGTJYIAMO7CLQVOZA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.034171487 (45m52.073744607s) [noise=true , address=45.91.116.5:4299 , piece_id=NGSDLHD5FZNMNHEZ7ZORWJ2CYGITBC653MWJNIVG7FPEZMICBR5Q, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.034924539 (45m52.074497659s) [noise=true , address=77.90.31.14:4663 , piece_id=PCCJUEMOBQXABFBC465Y4HT7JDYJKYKZDS6KH5CDKBRBIFKRSNWQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.059004085 (45m52.098577205s) [noise=true , address=45.91.116.12:4226 , piece_id=2NJ7KGH6WN7J5DGXLOAQLU5I7SYEV6BGMP2YP7AAM5RWGH5Q6IJA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.084407405 (45m52.123980525s) [noise=true , address=77.90.31.10:4519 , piece_id=K65NOSR2VT7SIHF5NC6XLNIBMV65FOQ6H6747DIVGTIJ374YQ2VQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.131868995 (45m52.171442135s) [noise=true , address=45.91.116.14:4189 , piece_id=DPXX4H72DZM75BSQBPD47XK2QBDL2VMXYNHNKLN3YWIUGQOSLYYQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.139780022 (45m52.179353202s) [noise=true , address=45.91.116.4:4939 , piece_id=P7UAZR7JTF53TDOO43EILUFQYW2ZWSH4XXHQSPXZ37XMYJG34YBQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.158799316 (45m52.198372436s) [noise=true , address=45.91.116.14:4191 , piece_id=C7HGF72ETJN6MJO5YA7527LIGMRXFHNT7ZJPAFWEJV5JLIMKQF7A, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.197157916 (45m52.236731036s) [noise=true , address=77.90.31.18:4805 , piece_id=DTF6UKXIML7VCJYZMCNUMOAIXC436BDQ3J7YOXWYSLV22SZAGEGA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.211520168 (45m52.251093288s) [noise=true , address=45.91.116.10:4082 , piece_id=CSWXLBIH75P7XANYYFVWZW273IV6XIVCKCYB5B4XB2BU2QFV3NVA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.309958463 (45m52.349531583s) [noise=true , address=45.91.116.4:4940 , piece_id=ZGP3LOKW7RIFIBJNKLCB3FXV72TMJRWBEWW7J6VWH7FMS7WE3CGQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.309985405 (45m52.349558525s) [seg_pos={0 0}, upload=659]: Segment reached optimal threshold of 13 pieces. Sep 6 14:14:16.310229742 (45m52.349802862s) [noise=true , address=77.90.31.19:4841 , piece_id=MWW64FHFPHG4L5BN42IIZYVE263OE7WEXGTCZQEDTX5VGLFYY7IA, seg_pos={0 0}, upload=659]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 14:14:16.310255922 (45m52.349829033s) [noise=true , address=45.91.116.8:4047 , piece_id=NWFXEENZKBXLMVA5T2XWRBGQUUSKRSHT4QKCEQA5Q5BTQDJHUCQA, seg_pos={0 0}, upload=659]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:33080->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:33080->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33080->45.91.116.8:4047: use of closed network connection Sep 6 14:14:16.310285108 (45m52.349858228s) [noise=true , address=45.91.116.7:4009 , piece_id=Y73WHM7ZZZTYQZ4QDGJ3WLPZB4I4BBWBARMNP4VAZPBCF7YIMZ4A, seg_pos={0 0}, upload=659]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:33034->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:33034->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33034->45.91.116.7:4009: use of closed network connection Sep 6 14:14:16.310296540 (45m52.34986966s) [seg_pos={0 0}, upload=659]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:14:16.310352697 (45m52.349925817s) [upload=659]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:14:16.310356124 (45m52.349929244s) [upload=659]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:14:16.616369589 (45m52.655942709s) [upload=659]: Done uploading segments... Sep 6 14:14:16.616490461 (45m52.656063581s) [upload=659]: Upload finished. err: <nil> Duration: 19.489487913s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 22 out of 60 is failed (threshold=3) [.XX......XX........XXX...XX...XX..XXX..XX..XX.....XX....X..X] duration: 30.002004371s FAIL (2025-09-06 14:14:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:14:13.283219758 (45m49.322792878s) [upload=659]: Starting upload Sep 6 14:14:13.283280274 (45m49.322853435s) [upload=659]: Done starting upload Sep 6 14:14:13.283345298 (45m49.322918418s) [upload=659]: Uploading segments... Sep 6 14:14:13.283587732 (45m49.323160842s) [upload=659]: Got next segment. Inline: false Sep 6 14:14:13.283640403 (45m49.323213513s) [upload=659]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:14:13.283646044 (45m49.323219164s) [upload=659]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:14:13.751750013 (45m49.791323133s) [seg_pos={0 0}, upload=659]: Begin upload segment... Sep 6 14:14:13.751788868 (45m49.791361988s) [seg_pos={0 0}, upload=659]: Done begin upload segment. Sep 6 14:14:13.751811311 (45m49.791384431s) [noise=true , address=77.90.31.10:4519 , piece_id=K65NOSR2VT7SIHF5NC6XLNIBMV65FOQ6H6747DIVGTIJ374YQ2VQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.751856537 (45m49.791429657s) [noise=true , address=45.91.116.4:4940 , piece_id=ZGP3LOKW7RIFIBJNKLCB3FXV72TMJRWBEWW7J6VWH7FMS7WE3CGQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752167342 (45m49.791740452s) [noise=true , address=77.90.31.18:4807 , piece_id=WQQJYDTZJCW3KX7DIYULZ7W2TAMCI5DMZQB6XEH7I7X2KZBBTWEA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752484548 (45m49.792057668s) [noise=true , address=45.91.116.7:4009 , piece_id=Y73WHM7ZZZTYQZ4QDGJ3WLPZB4I4BBWBARMNP4VAZPBCF7YIMZ4A, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752562708 (45m49.792135888s) [noise=true , address=45.91.116.14:4189 , piece_id=DPXX4H72DZM75BSQBPD47XK2QBDL2VMXYNHNKLN3YWIUGQOSLYYQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.752754856 (45m49.792327976s) [noise=true , address=45.91.116.10:4082 , piece_id=CSWXLBIH75P7XANYYFVWZW273IV6XIVCKCYB5B4XB2BU2QFV3NVA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753037867 (45m49.792610987s) [noise=true , address=45.91.116.9:4262 , piece_id=3Z3FV7DAS3G3Z6LVVPB2CBCPRZIVCG2NZ3BHQ5DGAGCX4YLAO52A, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753254622 (45m49.792827742s) [noise=true , address=77.90.31.18:4805 , piece_id=DTF6UKXIML7VCJYZMCNUMOAIXC436BDQ3J7YOXWYSLV22SZAGEGA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753468501 (45m49.793041611s) [noise=true , address=45.91.116.8:4048 , piece_id=3QI6WCPSLVBHVGGCX2UYILYJ5JOJTZL6HPMIGTJYIAMO7CLQVOZA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.753949412 (45m49.793522532s) [noise=true , address=45.91.116.4:4939 , piece_id=P7UAZR7JTF53TDOO43EILUFQYW2ZWSH4XXHQSPXZ37XMYJG34YBQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754168070 (45m49.79374118s) [noise=true , address=45.91.116.14:4191 , piece_id=C7HGF72ETJN6MJO5YA7527LIGMRXFHNT7ZJPAFWEJV5JLIMKQF7A, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754382671 (45m49.793955781s) [noise=true , address=45.91.116.12:4226 , piece_id=2NJ7KGH6WN7J5DGXLOAQLU5I7SYEV6BGMP2YP7AAM5RWGH5Q6IJA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754622750 (45m49.79419586s) [noise=true , address=77.90.31.19:4841 , piece_id=MWW64FHFPHG4L5BN42IIZYVE263OE7WEXGTCZQEDTX5VGLFYY7IA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.754843513 (45m49.794416633s) [noise=true , address=77.90.31.14:4663 , piece_id=PCCJUEMOBQXABFBC465Y4HT7JDYJKYKZDS6KH5CDKBRBIFKRSNWQ, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.755048144 (45m49.794621264s) [noise=true , address=45.91.116.8:4047 , piece_id=NWFXEENZKBXLMVA5T2XWRBGQUUSKRSHT4QKCEQA5Q5BTQDJHUCQA, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:13.755242225 (45m49.794815345s) [noise=true , address=45.91.116.5:4299 , piece_id=NGSDLHD5FZNMNHEZ7ZORWJ2CYGITBC653MWJNIVG7FPEZMICBR5Q, seg_pos={0 0}, upload=659]: Uploading piece... Sep 6 14:14:15.614631426 (45m51.654204546s) [upload=659]: Next returned nil segment Sep 6 14:14:15.614646394 (45m51.654219505s) [upload=659]: Waiting for error group managing segments... Sep 6 14:14:15.935223928 (45m51.974797048s) [noise=true , address=45.91.116.9:4262 , piece_id=3Z3FV7DAS3G3Z6LVVPB2CBCPRZIVCG2NZ3BHQ5DGAGCX4YLAO52A, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:15.955797296 (45m51.995370416s) [noise=true , address=77.90.31.18:4807 , piece_id=WQQJYDTZJCW3KX7DIYULZ7W2TAMCI5DMZQB6XEH7I7X2KZBBTWEA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:15.967768253 (45m52.007341373s) [noise=true , address=45.91.116.8:4048 , piece_id=3QI6WCPSLVBHVGGCX2UYILYJ5JOJTZL6HPMIGTJYIAMO7CLQVOZA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.034171487 (45m52.073744607s) [noise=true , address=45.91.116.5:4299 , piece_id=NGSDLHD5FZNMNHEZ7ZORWJ2CYGITBC653MWJNIVG7FPEZMICBR5Q, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.034924539 (45m52.074497659s) [noise=true , address=77.90.31.14:4663 , piece_id=PCCJUEMOBQXABFBC465Y4HT7JDYJKYKZDS6KH5CDKBRBIFKRSNWQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.059004085 (45m52.098577205s) [noise=true , address=45.91.116.12:4226 , piece_id=2NJ7KGH6WN7J5DGXLOAQLU5I7SYEV6BGMP2YP7AAM5RWGH5Q6IJA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.084407405 (45m52.123980525s) [noise=true , address=77.90.31.10:4519 , piece_id=K65NOSR2VT7SIHF5NC6XLNIBMV65FOQ6H6747DIVGTIJ374YQ2VQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.131868995 (45m52.171442135s) [noise=true , address=45.91.116.14:4189 , piece_id=DPXX4H72DZM75BSQBPD47XK2QBDL2VMXYNHNKLN3YWIUGQOSLYYQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.139780022 (45m52.179353202s) [noise=true , address=45.91.116.4:4939 , piece_id=P7UAZR7JTF53TDOO43EILUFQYW2ZWSH4XXHQSPXZ37XMYJG34YBQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.158799316 (45m52.198372436s) [noise=true , address=45.91.116.14:4191 , piece_id=C7HGF72ETJN6MJO5YA7527LIGMRXFHNT7ZJPAFWEJV5JLIMKQF7A, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.197157916 (45m52.236731036s) [noise=true , address=77.90.31.18:4805 , piece_id=DTF6UKXIML7VCJYZMCNUMOAIXC436BDQ3J7YOXWYSLV22SZAGEGA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.211520168 (45m52.251093288s) [noise=true , address=45.91.116.10:4082 , piece_id=CSWXLBIH75P7XANYYFVWZW273IV6XIVCKCYB5B4XB2BU2QFV3NVA, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.309958463 (45m52.349531583s) [noise=true , address=45.91.116.4:4940 , piece_id=ZGP3LOKW7RIFIBJNKLCB3FXV72TMJRWBEWW7J6VWH7FMS7WE3CGQ, seg_pos={0 0}, upload=659]: Done uploading piece. err: <nil> Sep 6 14:14:16.309985405 (45m52.349558525s) [seg_pos={0 0}, upload=659]: Segment reached optimal threshold of 13 pieces. Sep 6 14:14:16.310229742 (45m52.349802862s) [noise=true , address=77.90.31.19:4841 , piece_id=MWW64FHFPHG4L5BN42IIZYVE263OE7WEXGTCZQEDTX5VGLFYY7IA, seg_pos={0 0}, upload=659]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 14:14:16.310255922 (45m52.349829033s) [noise=true , address=45.91.116.8:4047 , piece_id=NWFXEENZKBXLMVA5T2XWRBGQUUSKRSHT4QKCEQA5Q5BTQDJHUCQA, seg_pos={0 0}, upload=659]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: write tcp 188.166.225.170:33080->45.91.116.8:4047: use of closed network connection; write tcp 188.166.225.170:33080->45.91.116.8:4047: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33080->45.91.116.8:4047: use of closed network connection Sep 6 14:14:16.310285108 (45m52.349858228s) [noise=true , address=45.91.116.7:4009 , piece_id=Y73WHM7ZZZTYQZ4QDGJ3WLPZB4I4BBWBARMNP4VAZPBCF7YIMZ4A, seg_pos={0 0}, upload=659]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1KaUBuZoi8DjagfvnxBSLBL1Mt69RdtNKn96oT7wFC8n2ppFDE): protocol: write tcp 188.166.225.170:33034->45.91.116.7:4009: use of closed network connection; write tcp 188.166.225.170:33034->45.91.116.7:4009: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33034->45.91.116.7:4009: use of closed network connection Sep 6 14:14:16.310296540 (45m52.34986966s) [seg_pos={0 0}, upload=659]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:14:16.310352697 (45m52.349925817s) [upload=659]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:14:16.310356124 (45m52.349929244s) [upload=659]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:14:16.616369589 (45m52.655942709s) [upload=659]: Done uploading segments... Sep 6 14:14:16.616490461 (45m52.656063581s) [upload=659]: Upload finished. err: <nil> Duration: 30.002022756s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [..XX......XX........XXX...XX...XX..XXX..XX..XX.....XX....X..] duration: 16.582432175s FAIL (2025-09-06 14:13:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:11:26.156436794 (43m2.196009944s) [upload=622]: Starting upload Sep 6 14:11:26.156543658 (43m2.196116808s) [upload=622]: Done starting upload Sep 6 14:11:26.156603322 (43m2.196176472s) [upload=622]: Uploading segments... Sep 6 14:11:26.156921802 (43m2.196494922s) [upload=622]: Got next segment. Inline: false Sep 6 14:11:26.156955005 (43m2.196528125s) [upload=622]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:11:26.156960355 (43m2.196533475s) [upload=622]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:11:26.698970276 (43m2.738543396s) [seg_pos={0 0}, upload=622]: Begin upload segment... Sep 6 14:11:26.699013669 (43m2.738586789s) [seg_pos={0 0}, upload=622]: Done begin upload segment. Sep 6 14:11:26.699030712 (43m2.738603832s) [noise=true , address=45.91.116.11:4118 , piece_id=LY44OTQEXAKWAO2VWDRS6LYEGZN3AWIBGVRRGLTNWNITNNPP2PSA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699041302 (43m2.738614422s) [noise=true , address=77.90.31.18:4805 , piece_id=NGDUOMB75WJSBWKXOB2WN2YVI37L5ZXYUDW3FBDLQ5AVYQNX2E3A, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699098932 (43m2.738672052s) [noise=true , address=45.91.116.12:4227 , piece_id=ONVH6TSH3IB6ZPBB5YCUYISNYAZOQVBOSKZJU5VRKB5G76ZUP3SQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699399788 (43m2.738972908s) [noise=true , address=77.90.31.12:4592 , piece_id=WNLWTQYINCIZKJNP56TAJYZUEBQQ4AZ2TECS6IMVECP7SPRZ43HA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699478007 (43m2.739051128s) [noise=true , address=77.90.31.16:4733 , piece_id=ZYAPQ55OZ4YKDRIQVJXJD46RG2N26T3AXAQNUHQCHRASKEWHW4YA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699712056 (43m2.739285176s) [noise=true , address=45.91.116.10:4082 , piece_id=S7TBI2K2YKPI3M63SFC454OM6WFFIF2E42UTYMZKEKLGZHNV7WJQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699968055 (43m2.739541175s) [noise=true , address=45.91.116.6:4973 , piece_id=IOQKGWX6QXQ5YVDVOPUHLJYJUMGLNICOGNFVUF4ZSN2WMAE2654Q, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700287156 (43m2.739860276s) [noise=true , address=77.90.31.14:4661 , piece_id=LFNFTI7I7JZUD5MNMRLU6K4K6W5KDBYQRYHDZT6KXPRK4DAI57HQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700554247 (43m2.740127367s) [noise=true , address=77.90.31.12:4589 , piece_id=MRC5EQ6TVVCQA3TGTX7RTBTLLXDLLKRWS3WCBDAZIIHFSCDJ4CPQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700859582 (43m2.740432712s) [noise=true , address=45.91.116.11:4117 , piece_id=VILMQLI5IM5LWVM3XIPTXFRKRUUFGZBXGHZ47KG2UIJHTHAG7VWA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701058512 (43m2.740631632s) [noise=true , address=45.91.116.13:4154 , piece_id=Q4CNGNOO6ERAVZZI46ZM5TFFN3EPV3W6FMKQKEOR4AO5OIOIXPTA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701275076 (43m2.740848197s) [noise=true , address=77.90.31.18:4806 , piece_id=7VHTHDIJWUD2L4FJCXNZNHZGLQ5O5A7RSMLVM5VSDRW3LWYRPHEA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701468256 (43m2.741041376s) [noise=true , address=45.91.116.14:4189 , piece_id=XRC2SYVP6JXD653KIFR7Y7X6UT32Z7SJ6HR33RFSNNGFWFNHAQWQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701658009 (43m2.741231129s) [noise=true , address=45.91.116.11:4119 , piece_id=7ZNOZGACVRJNZMHRAU4D6267ZVDPW3TRIDBQLVHLULFMOOL23AHA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701884232 (43m2.741457352s) [noise=true , address=45.91.116.4:4938 , piece_id=BXGYBHSBDHIRRWPPE7I2XRW3X7EXV5YNLDAXPEKV4ARRBGCM4XTQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.702073454 (43m2.741646584s) [noise=true , address=45.91.116.8:4045 , piece_id=26OEWKAFGNSZIZ56L6XCE4AC2ORTMHITEETVWXBBIAZOLMJIS32Q, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:28.545405048 (43m4.584978168s) [upload=622]: Next returned nil segment Sep 6 14:11:28.545423043 (43m4.584996163s) [upload=622]: Waiting for error group managing segments... Sep 6 14:11:28.877920115 (43m4.917493225s) [noise=true , address=45.91.116.8:4045 , piece_id=26OEWKAFGNSZIZ56L6XCE4AC2ORTMHITEETVWXBBIAZOLMJIS32Q, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.897359041 (43m4.936932161s) [noise=true , address=45.91.116.6:4973 , piece_id=IOQKGWX6QXQ5YVDVOPUHLJYJUMGLNICOGNFVUF4ZSN2WMAE2654Q, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.897794004 (43m4.937367124s) [noise=true , address=45.91.116.11:4119 , piece_id=7ZNOZGACVRJNZMHRAU4D6267ZVDPW3TRIDBQLVHLULFMOOL23AHA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.910505728 (43m4.950078848s) [noise=true , address=77.90.31.18:4806 , piece_id=7VHTHDIJWUD2L4FJCXNZNHZGLQ5O5A7RSMLVM5VSDRW3LWYRPHEA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.931114373 (43m4.970687493s) [noise=true , address=77.90.31.16:4733 , piece_id=ZYAPQ55OZ4YKDRIQVJXJD46RG2N26T3AXAQNUHQCHRASKEWHW4YA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.942495481 (43m4.982068601s) [noise=true , address=45.91.116.11:4118 , piece_id=LY44OTQEXAKWAO2VWDRS6LYEGZN3AWIBGVRRGLTNWNITNNPP2PSA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.945671949 (43m4.985245069s) [noise=true , address=45.91.116.12:4227 , piece_id=ONVH6TSH3IB6ZPBB5YCUYISNYAZOQVBOSKZJU5VRKB5G76ZUP3SQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.958720488 (43m4.998293609s) [noise=true , address=45.91.116.4:4938 , piece_id=BXGYBHSBDHIRRWPPE7I2XRW3X7EXV5YNLDAXPEKV4ARRBGCM4XTQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.015279164 (43m5.054852405s) [noise=true , address=45.91.116.14:4189 , piece_id=XRC2SYVP6JXD653KIFR7Y7X6UT32Z7SJ6HR33RFSNNGFWFNHAQWQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.027017476 (43m5.066590596s) [noise=true , address=45.91.116.13:4154 , piece_id=Q4CNGNOO6ERAVZZI46ZM5TFFN3EPV3W6FMKQKEOR4AO5OIOIXPTA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.061633345 (43m5.101206455s) [noise=true , address=45.91.116.11:4117 , piece_id=VILMQLI5IM5LWVM3XIPTXFRKRUUFGZBXGHZ47KG2UIJHTHAG7VWA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.177711703 (43m5.217284823s) [noise=true , address=45.91.116.10:4082 , piece_id=S7TBI2K2YKPI3M63SFC454OM6WFFIF2E42UTYMZKEKLGZHNV7WJQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.670607101 (43m5.710180222s) [noise=true , address=77.90.31.12:4589 , piece_id=MRC5EQ6TVVCQA3TGTX7RTBTLLXDLLKRWS3WCBDAZIIHFSCDJ4CPQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.670639072 (43m5.710212243s) [seg_pos={0 0}, upload=622]: Segment reached optimal threshold of 13 pieces. Sep 6 14:11:29.670817925 (43m5.710391075s) [noise=true , address=77.90.31.12:4592 , piece_id=WNLWTQYINCIZKJNP56TAJYZUEBQQ4AZ2TECS6IMVECP7SPRZ43HA, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670911784 (43m5.710484904s) [noise=true , address=77.90.31.18:4805 , piece_id=NGDUOMB75WJSBWKXOB2WN2YVI37L5ZXYUDW3FBDLQ5AVYQNX2E3A, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670950449 (43m5.710523559s) [noise=true , address=77.90.31.14:4661 , piece_id=LFNFTI7I7JZUD5MNMRLU6K4K6W5KDBYQRYHDZT6KXPRK4DAI57HQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670975667 (43m5.710548787s) [seg_pos={0 0}, upload=622]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:11:29.671059908 (43m5.710633028s) [upload=622]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:11:29.671064888 (43m5.710638008s) [upload=622]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:11:30.358688353 (43m6.398261473s) [upload=622]: Done uploading segments... Sep 6 14:11:30.358707250 (43m6.39828037s) [upload=622]: Upload finished. err: <nil> Duration: 16.582458214s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [...XX......XX........XXX...XX...XX..XXX..XX..XX.....XX....X.] duration: 13.847131103s FAIL (2025-09-06 14:12:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:11:26.156436794 (43m2.196009944s) [upload=622]: Starting upload Sep 6 14:11:26.156543658 (43m2.196116808s) [upload=622]: Done starting upload Sep 6 14:11:26.156603322 (43m2.196176472s) [upload=622]: Uploading segments... Sep 6 14:11:26.156921802 (43m2.196494922s) [upload=622]: Got next segment. Inline: false Sep 6 14:11:26.156955005 (43m2.196528125s) [upload=622]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:11:26.156960355 (43m2.196533475s) [upload=622]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:11:26.698970276 (43m2.738543396s) [seg_pos={0 0}, upload=622]: Begin upload segment... Sep 6 14:11:26.699013669 (43m2.738586789s) [seg_pos={0 0}, upload=622]: Done begin upload segment. Sep 6 14:11:26.699030712 (43m2.738603832s) [noise=true , address=45.91.116.11:4118 , piece_id=LY44OTQEXAKWAO2VWDRS6LYEGZN3AWIBGVRRGLTNWNITNNPP2PSA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699041302 (43m2.738614422s) [noise=true , address=77.90.31.18:4805 , piece_id=NGDUOMB75WJSBWKXOB2WN2YVI37L5ZXYUDW3FBDLQ5AVYQNX2E3A, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699098932 (43m2.738672052s) [noise=true , address=45.91.116.12:4227 , piece_id=ONVH6TSH3IB6ZPBB5YCUYISNYAZOQVBOSKZJU5VRKB5G76ZUP3SQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699399788 (43m2.738972908s) [noise=true , address=77.90.31.12:4592 , piece_id=WNLWTQYINCIZKJNP56TAJYZUEBQQ4AZ2TECS6IMVECP7SPRZ43HA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699478007 (43m2.739051128s) [noise=true , address=77.90.31.16:4733 , piece_id=ZYAPQ55OZ4YKDRIQVJXJD46RG2N26T3AXAQNUHQCHRASKEWHW4YA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699712056 (43m2.739285176s) [noise=true , address=45.91.116.10:4082 , piece_id=S7TBI2K2YKPI3M63SFC454OM6WFFIF2E42UTYMZKEKLGZHNV7WJQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699968055 (43m2.739541175s) [noise=true , address=45.91.116.6:4973 , piece_id=IOQKGWX6QXQ5YVDVOPUHLJYJUMGLNICOGNFVUF4ZSN2WMAE2654Q, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700287156 (43m2.739860276s) [noise=true , address=77.90.31.14:4661 , piece_id=LFNFTI7I7JZUD5MNMRLU6K4K6W5KDBYQRYHDZT6KXPRK4DAI57HQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700554247 (43m2.740127367s) [noise=true , address=77.90.31.12:4589 , piece_id=MRC5EQ6TVVCQA3TGTX7RTBTLLXDLLKRWS3WCBDAZIIHFSCDJ4CPQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700859582 (43m2.740432712s) [noise=true , address=45.91.116.11:4117 , piece_id=VILMQLI5IM5LWVM3XIPTXFRKRUUFGZBXGHZ47KG2UIJHTHAG7VWA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701058512 (43m2.740631632s) [noise=true , address=45.91.116.13:4154 , piece_id=Q4CNGNOO6ERAVZZI46ZM5TFFN3EPV3W6FMKQKEOR4AO5OIOIXPTA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701275076 (43m2.740848197s) [noise=true , address=77.90.31.18:4806 , piece_id=7VHTHDIJWUD2L4FJCXNZNHZGLQ5O5A7RSMLVM5VSDRW3LWYRPHEA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701468256 (43m2.741041376s) [noise=true , address=45.91.116.14:4189 , piece_id=XRC2SYVP6JXD653KIFR7Y7X6UT32Z7SJ6HR33RFSNNGFWFNHAQWQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701658009 (43m2.741231129s) [noise=true , address=45.91.116.11:4119 , piece_id=7ZNOZGACVRJNZMHRAU4D6267ZVDPW3TRIDBQLVHLULFMOOL23AHA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701884232 (43m2.741457352s) [noise=true , address=45.91.116.4:4938 , piece_id=BXGYBHSBDHIRRWPPE7I2XRW3X7EXV5YNLDAXPEKV4ARRBGCM4XTQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.702073454 (43m2.741646584s) [noise=true , address=45.91.116.8:4045 , piece_id=26OEWKAFGNSZIZ56L6XCE4AC2ORTMHITEETVWXBBIAZOLMJIS32Q, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:28.545405048 (43m4.584978168s) [upload=622]: Next returned nil segment Sep 6 14:11:28.545423043 (43m4.584996163s) [upload=622]: Waiting for error group managing segments... Sep 6 14:11:28.877920115 (43m4.917493225s) [noise=true , address=45.91.116.8:4045 , piece_id=26OEWKAFGNSZIZ56L6XCE4AC2ORTMHITEETVWXBBIAZOLMJIS32Q, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.897359041 (43m4.936932161s) [noise=true , address=45.91.116.6:4973 , piece_id=IOQKGWX6QXQ5YVDVOPUHLJYJUMGLNICOGNFVUF4ZSN2WMAE2654Q, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.897794004 (43m4.937367124s) [noise=true , address=45.91.116.11:4119 , piece_id=7ZNOZGACVRJNZMHRAU4D6267ZVDPW3TRIDBQLVHLULFMOOL23AHA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.910505728 (43m4.950078848s) [noise=true , address=77.90.31.18:4806 , piece_id=7VHTHDIJWUD2L4FJCXNZNHZGLQ5O5A7RSMLVM5VSDRW3LWYRPHEA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.931114373 (43m4.970687493s) [noise=true , address=77.90.31.16:4733 , piece_id=ZYAPQ55OZ4YKDRIQVJXJD46RG2N26T3AXAQNUHQCHRASKEWHW4YA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.942495481 (43m4.982068601s) [noise=true , address=45.91.116.11:4118 , piece_id=LY44OTQEXAKWAO2VWDRS6LYEGZN3AWIBGVRRGLTNWNITNNPP2PSA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.945671949 (43m4.985245069s) [noise=true , address=45.91.116.12:4227 , piece_id=ONVH6TSH3IB6ZPBB5YCUYISNYAZOQVBOSKZJU5VRKB5G76ZUP3SQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.958720488 (43m4.998293609s) [noise=true , address=45.91.116.4:4938 , piece_id=BXGYBHSBDHIRRWPPE7I2XRW3X7EXV5YNLDAXPEKV4ARRBGCM4XTQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.015279164 (43m5.054852405s) [noise=true , address=45.91.116.14:4189 , piece_id=XRC2SYVP6JXD653KIFR7Y7X6UT32Z7SJ6HR33RFSNNGFWFNHAQWQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.027017476 (43m5.066590596s) [noise=true , address=45.91.116.13:4154 , piece_id=Q4CNGNOO6ERAVZZI46ZM5TFFN3EPV3W6FMKQKEOR4AO5OIOIXPTA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.061633345 (43m5.101206455s) [noise=true , address=45.91.116.11:4117 , piece_id=VILMQLI5IM5LWVM3XIPTXFRKRUUFGZBXGHZ47KG2UIJHTHAG7VWA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.177711703 (43m5.217284823s) [noise=true , address=45.91.116.10:4082 , piece_id=S7TBI2K2YKPI3M63SFC454OM6WFFIF2E42UTYMZKEKLGZHNV7WJQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.670607101 (43m5.710180222s) [noise=true , address=77.90.31.12:4589 , piece_id=MRC5EQ6TVVCQA3TGTX7RTBTLLXDLLKRWS3WCBDAZIIHFSCDJ4CPQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.670639072 (43m5.710212243s) [seg_pos={0 0}, upload=622]: Segment reached optimal threshold of 13 pieces. Sep 6 14:11:29.670817925 (43m5.710391075s) [noise=true , address=77.90.31.12:4592 , piece_id=WNLWTQYINCIZKJNP56TAJYZUEBQQ4AZ2TECS6IMVECP7SPRZ43HA, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670911784 (43m5.710484904s) [noise=true , address=77.90.31.18:4805 , piece_id=NGDUOMB75WJSBWKXOB2WN2YVI37L5ZXYUDW3FBDLQ5AVYQNX2E3A, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670950449 (43m5.710523559s) [noise=true , address=77.90.31.14:4661 , piece_id=LFNFTI7I7JZUD5MNMRLU6K4K6W5KDBYQRYHDZT6KXPRK4DAI57HQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670975667 (43m5.710548787s) [seg_pos={0 0}, upload=622]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:11:29.671059908 (43m5.710633028s) [upload=622]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:11:29.671064888 (43m5.710638008s) [upload=622]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:11:30.358688353 (43m6.398261473s) [upload=622]: Done uploading segments... Sep 6 14:11:30.358707250 (43m6.39828037s) [upload=622]: Upload finished. err: <nil> Duration: 13.847157153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 21 out of 60 is failed (threshold=3) [....XX......XX........XXX...XX...XX..XXX..XX..XX.....XX....X] duration: 30.001411326s FAIL (2025-09-06 14:11:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:11:26.156436794 (43m2.196009944s) [upload=622]: Starting upload Sep 6 14:11:26.156543658 (43m2.196116808s) [upload=622]: Done starting upload Sep 6 14:11:26.156603322 (43m2.196176472s) [upload=622]: Uploading segments... Sep 6 14:11:26.156921802 (43m2.196494922s) [upload=622]: Got next segment. Inline: false Sep 6 14:11:26.156955005 (43m2.196528125s) [upload=622]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:11:26.156960355 (43m2.196533475s) [upload=622]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:11:26.698970276 (43m2.738543396s) [seg_pos={0 0}, upload=622]: Begin upload segment... Sep 6 14:11:26.699013669 (43m2.738586789s) [seg_pos={0 0}, upload=622]: Done begin upload segment. Sep 6 14:11:26.699030712 (43m2.738603832s) [noise=true , address=45.91.116.11:4118 , piece_id=LY44OTQEXAKWAO2VWDRS6LYEGZN3AWIBGVRRGLTNWNITNNPP2PSA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699041302 (43m2.738614422s) [noise=true , address=77.90.31.18:4805 , piece_id=NGDUOMB75WJSBWKXOB2WN2YVI37L5ZXYUDW3FBDLQ5AVYQNX2E3A, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699098932 (43m2.738672052s) [noise=true , address=45.91.116.12:4227 , piece_id=ONVH6TSH3IB6ZPBB5YCUYISNYAZOQVBOSKZJU5VRKB5G76ZUP3SQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699399788 (43m2.738972908s) [noise=true , address=77.90.31.12:4592 , piece_id=WNLWTQYINCIZKJNP56TAJYZUEBQQ4AZ2TECS6IMVECP7SPRZ43HA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699478007 (43m2.739051128s) [noise=true , address=77.90.31.16:4733 , piece_id=ZYAPQ55OZ4YKDRIQVJXJD46RG2N26T3AXAQNUHQCHRASKEWHW4YA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699712056 (43m2.739285176s) [noise=true , address=45.91.116.10:4082 , piece_id=S7TBI2K2YKPI3M63SFC454OM6WFFIF2E42UTYMZKEKLGZHNV7WJQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.699968055 (43m2.739541175s) [noise=true , address=45.91.116.6:4973 , piece_id=IOQKGWX6QXQ5YVDVOPUHLJYJUMGLNICOGNFVUF4ZSN2WMAE2654Q, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700287156 (43m2.739860276s) [noise=true , address=77.90.31.14:4661 , piece_id=LFNFTI7I7JZUD5MNMRLU6K4K6W5KDBYQRYHDZT6KXPRK4DAI57HQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700554247 (43m2.740127367s) [noise=true , address=77.90.31.12:4589 , piece_id=MRC5EQ6TVVCQA3TGTX7RTBTLLXDLLKRWS3WCBDAZIIHFSCDJ4CPQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.700859582 (43m2.740432712s) [noise=true , address=45.91.116.11:4117 , piece_id=VILMQLI5IM5LWVM3XIPTXFRKRUUFGZBXGHZ47KG2UIJHTHAG7VWA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701058512 (43m2.740631632s) [noise=true , address=45.91.116.13:4154 , piece_id=Q4CNGNOO6ERAVZZI46ZM5TFFN3EPV3W6FMKQKEOR4AO5OIOIXPTA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701275076 (43m2.740848197s) [noise=true , address=77.90.31.18:4806 , piece_id=7VHTHDIJWUD2L4FJCXNZNHZGLQ5O5A7RSMLVM5VSDRW3LWYRPHEA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701468256 (43m2.741041376s) [noise=true , address=45.91.116.14:4189 , piece_id=XRC2SYVP6JXD653KIFR7Y7X6UT32Z7SJ6HR33RFSNNGFWFNHAQWQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701658009 (43m2.741231129s) [noise=true , address=45.91.116.11:4119 , piece_id=7ZNOZGACVRJNZMHRAU4D6267ZVDPW3TRIDBQLVHLULFMOOL23AHA, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.701884232 (43m2.741457352s) [noise=true , address=45.91.116.4:4938 , piece_id=BXGYBHSBDHIRRWPPE7I2XRW3X7EXV5YNLDAXPEKV4ARRBGCM4XTQ, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:26.702073454 (43m2.741646584s) [noise=true , address=45.91.116.8:4045 , piece_id=26OEWKAFGNSZIZ56L6XCE4AC2ORTMHITEETVWXBBIAZOLMJIS32Q, seg_pos={0 0}, upload=622]: Uploading piece... Sep 6 14:11:28.545405048 (43m4.584978168s) [upload=622]: Next returned nil segment Sep 6 14:11:28.545423043 (43m4.584996163s) [upload=622]: Waiting for error group managing segments... Sep 6 14:11:28.877920115 (43m4.917493225s) [noise=true , address=45.91.116.8:4045 , piece_id=26OEWKAFGNSZIZ56L6XCE4AC2ORTMHITEETVWXBBIAZOLMJIS32Q, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.897359041 (43m4.936932161s) [noise=true , address=45.91.116.6:4973 , piece_id=IOQKGWX6QXQ5YVDVOPUHLJYJUMGLNICOGNFVUF4ZSN2WMAE2654Q, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.897794004 (43m4.937367124s) [noise=true , address=45.91.116.11:4119 , piece_id=7ZNOZGACVRJNZMHRAU4D6267ZVDPW3TRIDBQLVHLULFMOOL23AHA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.910505728 (43m4.950078848s) [noise=true , address=77.90.31.18:4806 , piece_id=7VHTHDIJWUD2L4FJCXNZNHZGLQ5O5A7RSMLVM5VSDRW3LWYRPHEA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.931114373 (43m4.970687493s) [noise=true , address=77.90.31.16:4733 , piece_id=ZYAPQ55OZ4YKDRIQVJXJD46RG2N26T3AXAQNUHQCHRASKEWHW4YA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.942495481 (43m4.982068601s) [noise=true , address=45.91.116.11:4118 , piece_id=LY44OTQEXAKWAO2VWDRS6LYEGZN3AWIBGVRRGLTNWNITNNPP2PSA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.945671949 (43m4.985245069s) [noise=true , address=45.91.116.12:4227 , piece_id=ONVH6TSH3IB6ZPBB5YCUYISNYAZOQVBOSKZJU5VRKB5G76ZUP3SQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:28.958720488 (43m4.998293609s) [noise=true , address=45.91.116.4:4938 , piece_id=BXGYBHSBDHIRRWPPE7I2XRW3X7EXV5YNLDAXPEKV4ARRBGCM4XTQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.015279164 (43m5.054852405s) [noise=true , address=45.91.116.14:4189 , piece_id=XRC2SYVP6JXD653KIFR7Y7X6UT32Z7SJ6HR33RFSNNGFWFNHAQWQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.027017476 (43m5.066590596s) [noise=true , address=45.91.116.13:4154 , piece_id=Q4CNGNOO6ERAVZZI46ZM5TFFN3EPV3W6FMKQKEOR4AO5OIOIXPTA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.061633345 (43m5.101206455s) [noise=true , address=45.91.116.11:4117 , piece_id=VILMQLI5IM5LWVM3XIPTXFRKRUUFGZBXGHZ47KG2UIJHTHAG7VWA, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.177711703 (43m5.217284823s) [noise=true , address=45.91.116.10:4082 , piece_id=S7TBI2K2YKPI3M63SFC454OM6WFFIF2E42UTYMZKEKLGZHNV7WJQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.670607101 (43m5.710180222s) [noise=true , address=77.90.31.12:4589 , piece_id=MRC5EQ6TVVCQA3TGTX7RTBTLLXDLLKRWS3WCBDAZIIHFSCDJ4CPQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: <nil> Sep 6 14:11:29.670639072 (43m5.710212243s) [seg_pos={0 0}, upload=622]: Segment reached optimal threshold of 13 pieces. Sep 6 14:11:29.670817925 (43m5.710391075s) [noise=true , address=77.90.31.12:4592 , piece_id=WNLWTQYINCIZKJNP56TAJYZUEBQQ4AZ2TECS6IMVECP7SPRZ43HA, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670911784 (43m5.710484904s) [noise=true , address=77.90.31.18:4805 , piece_id=NGDUOMB75WJSBWKXOB2WN2YVI37L5ZXYUDW3FBDLQ5AVYQNX2E3A, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ExMgyeH6s1VMkWatpEqPQwMhX4aGJeK6AWn98kD4f6fQV7rcT): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670950449 (43m5.710523559s) [noise=true , address=77.90.31.14:4661 , piece_id=LFNFTI7I7JZUD5MNMRLU6K4K6W5KDBYQRYHDZT6KXPRK4DAI57HQ, seg_pos={0 0}, upload=622]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faSN2qTnzvZquF4ekHA6PX5tDSAyrVTkWuy1Z3FxgByxZ4jue): protocol: expected piece hash; context canceled; EOF Sep 6 14:11:29.670975667 (43m5.710548787s) [seg_pos={0 0}, upload=622]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:11:29.671059908 (43m5.710633028s) [upload=622]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:11:29.671064888 (43m5.710638008s) [upload=622]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:11:30.358688353 (43m6.398261473s) [upload=622]: Done uploading segments... Sep 6 14:11:30.358707250 (43m6.39828037s) [upload=622]: Upload finished. err: <nil> Duration: 30.001422077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.....XX......XX........XXX...XX...XX..XXX..XX..XX.....XX....] duration: 28.808290319s FAIL (2025-09-06 14:10:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:07:58.816734823 (39m34.856307943s) [upload=573]: Starting upload Sep 6 14:07:58.816793234 (39m34.856366354s) [upload=573]: Done starting upload Sep 6 14:07:58.816878788 (39m34.856451908s) [upload=573]: Uploading segments... Sep 6 14:07:58.817014688 (39m34.856587808s) [upload=573]: Got next segment. Inline: false Sep 6 14:07:58.817038183 (39m34.856611303s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:07:58.817042611 (39m34.856615731s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:07:59.090619398 (39m35.130192558s) [seg_pos={0 0}, upload=573]: Begin upload segment... Sep 6 14:07:59.090671286 (39m35.130244406s) [seg_pos={0 0}, upload=573]: Done begin upload segment. Sep 6 14:07:59.090694731 (39m35.130267851s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090700253 (39m35.130273373s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090737634 (39m35.130310744s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090785436 (39m35.130358556s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091146606 (39m35.130719727s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091511394 (39m35.131084514s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091781421 (39m35.131354541s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092038142 (39m35.131611262s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092299963 (39m35.131873073s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092543239 (39m35.132116359s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092745115 (39m35.132318235s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092944447 (39m35.132517567s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093145752 (39m35.132718872s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093382265 (39m35.132955385s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093579211 (39m35.133152331s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093764656 (39m35.133337776s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:08:00.937684886 (39m36.977258016s) [upload=573]: Next returned nil segment Sep 6 14:08:00.937711106 (39m36.977284226s) [upload=573]: Waiting for error group managing segments... Sep 6 14:08:01.229627256 (39m37.269200376s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.239225592 (39m37.278798712s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.247313068 (39m37.286886218s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.249756584 (39m37.289329704s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.251873975 (39m37.291447105s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.274644587 (39m37.314217717s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.285486915 (39m37.325060035s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.359873446 (39m37.399446566s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.415375370 (39m37.45494849s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.432187200 (39m37.47176032s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.449428201 (39m37.489001321s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.529655178 (39m37.569228298s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716515189 (39m37.756088309s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716548723 (39m37.756121853s) [seg_pos={0 0}, upload=573]: Segment reached optimal threshold of 13 pieces. Sep 6 14:08:01.716873925 (39m37.756447055s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection Sep 6 14:08:01.716895096 (39m37.756468216s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection Sep 6 14:08:01.716949470 (39m37.75652259s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection Sep 6 14:08:01.716976963 (39m37.756550083s) [seg_pos={0 0}, upload=573]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:08:01.717065301 (39m37.756638421s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:08:01.717069790 (39m37.75664292s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:08:02.058321383 (39m38.097894514s) [upload=573]: Done uploading segments... Sep 6 14:08:02.058345169 (39m38.097918289s) [upload=573]: Upload finished. err: <nil> Duration: 28.808310587s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [......XX......XX........XXX...XX...XX..XXX..XX..XX.....XX...] duration: 9.483798519s FAIL (2025-09-06 14:09:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:07:58.816734823 (39m34.856307943s) [upload=573]: Starting upload Sep 6 14:07:58.816793234 (39m34.856366354s) [upload=573]: Done starting upload Sep 6 14:07:58.816878788 (39m34.856451908s) [upload=573]: Uploading segments... Sep 6 14:07:58.817014688 (39m34.856587808s) [upload=573]: Got next segment. Inline: false Sep 6 14:07:58.817038183 (39m34.856611303s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:07:58.817042611 (39m34.856615731s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:07:59.090619398 (39m35.130192558s) [seg_pos={0 0}, upload=573]: Begin upload segment... Sep 6 14:07:59.090671286 (39m35.130244406s) [seg_pos={0 0}, upload=573]: Done begin upload segment. Sep 6 14:07:59.090694731 (39m35.130267851s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090700253 (39m35.130273373s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090737634 (39m35.130310744s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090785436 (39m35.130358556s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091146606 (39m35.130719727s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091511394 (39m35.131084514s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091781421 (39m35.131354541s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092038142 (39m35.131611262s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092299963 (39m35.131873073s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092543239 (39m35.132116359s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092745115 (39m35.132318235s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092944447 (39m35.132517567s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093145752 (39m35.132718872s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093382265 (39m35.132955385s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093579211 (39m35.133152331s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093764656 (39m35.133337776s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:08:00.937684886 (39m36.977258016s) [upload=573]: Next returned nil segment Sep 6 14:08:00.937711106 (39m36.977284226s) [upload=573]: Waiting for error group managing segments... Sep 6 14:08:01.229627256 (39m37.269200376s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.239225592 (39m37.278798712s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.247313068 (39m37.286886218s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.249756584 (39m37.289329704s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.251873975 (39m37.291447105s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.274644587 (39m37.314217717s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.285486915 (39m37.325060035s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.359873446 (39m37.399446566s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.415375370 (39m37.45494849s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.432187200 (39m37.47176032s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.449428201 (39m37.489001321s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.529655178 (39m37.569228298s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716515189 (39m37.756088309s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716548723 (39m37.756121853s) [seg_pos={0 0}, upload=573]: Segment reached optimal threshold of 13 pieces. Sep 6 14:08:01.716873925 (39m37.756447055s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection Sep 6 14:08:01.716895096 (39m37.756468216s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection Sep 6 14:08:01.716949470 (39m37.75652259s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection Sep 6 14:08:01.716976963 (39m37.756550083s) [seg_pos={0 0}, upload=573]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:08:01.717065301 (39m37.756638421s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:08:01.717069790 (39m37.75664292s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:08:02.058321383 (39m38.097894514s) [upload=573]: Done uploading segments... Sep 6 14:08:02.058345169 (39m38.097918289s) [upload=573]: Upload finished. err: <nil> Duration: 9.483813508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.......XX......XX........XXX...XX...XX..XXX..XX..XX.....XX..] duration: 26.212899449s FAIL (2025-09-06 14:09:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:07:58.816734823 (39m34.856307943s) [upload=573]: Starting upload Sep 6 14:07:58.816793234 (39m34.856366354s) [upload=573]: Done starting upload Sep 6 14:07:58.816878788 (39m34.856451908s) [upload=573]: Uploading segments... Sep 6 14:07:58.817014688 (39m34.856587808s) [upload=573]: Got next segment. Inline: false Sep 6 14:07:58.817038183 (39m34.856611303s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:07:58.817042611 (39m34.856615731s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:07:59.090619398 (39m35.130192558s) [seg_pos={0 0}, upload=573]: Begin upload segment... Sep 6 14:07:59.090671286 (39m35.130244406s) [seg_pos={0 0}, upload=573]: Done begin upload segment. Sep 6 14:07:59.090694731 (39m35.130267851s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090700253 (39m35.130273373s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090737634 (39m35.130310744s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090785436 (39m35.130358556s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091146606 (39m35.130719727s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091511394 (39m35.131084514s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091781421 (39m35.131354541s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092038142 (39m35.131611262s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092299963 (39m35.131873073s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092543239 (39m35.132116359s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092745115 (39m35.132318235s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092944447 (39m35.132517567s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093145752 (39m35.132718872s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093382265 (39m35.132955385s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093579211 (39m35.133152331s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093764656 (39m35.133337776s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:08:00.937684886 (39m36.977258016s) [upload=573]: Next returned nil segment Sep 6 14:08:00.937711106 (39m36.977284226s) [upload=573]: Waiting for error group managing segments... Sep 6 14:08:01.229627256 (39m37.269200376s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.239225592 (39m37.278798712s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.247313068 (39m37.286886218s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.249756584 (39m37.289329704s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.251873975 (39m37.291447105s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.274644587 (39m37.314217717s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.285486915 (39m37.325060035s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.359873446 (39m37.399446566s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.415375370 (39m37.45494849s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.432187200 (39m37.47176032s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.449428201 (39m37.489001321s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.529655178 (39m37.569228298s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716515189 (39m37.756088309s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716548723 (39m37.756121853s) [seg_pos={0 0}, upload=573]: Segment reached optimal threshold of 13 pieces. Sep 6 14:08:01.716873925 (39m37.756447055s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection Sep 6 14:08:01.716895096 (39m37.756468216s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection Sep 6 14:08:01.716949470 (39m37.75652259s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection Sep 6 14:08:01.716976963 (39m37.756550083s) [seg_pos={0 0}, upload=573]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:08:01.717065301 (39m37.756638421s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:08:01.717069790 (39m37.75664292s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:08:02.058321383 (39m38.097894514s) [upload=573]: Done uploading segments... Sep 6 14:08:02.058345169 (39m38.097918289s) [upload=573]: Upload finished. err: <nil> Duration: 26.212928726s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [........XX......XX........XXX...XX...XX..XXX..XX..XX.....XX.] duration: 15.225769704s FAIL (2025-09-06 14:08:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:07:58.816734823 (39m34.856307943s) [upload=573]: Starting upload Sep 6 14:07:58.816793234 (39m34.856366354s) [upload=573]: Done starting upload Sep 6 14:07:58.816878788 (39m34.856451908s) [upload=573]: Uploading segments... Sep 6 14:07:58.817014688 (39m34.856587808s) [upload=573]: Got next segment. Inline: false Sep 6 14:07:58.817038183 (39m34.856611303s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:07:58.817042611 (39m34.856615731s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:07:59.090619398 (39m35.130192558s) [seg_pos={0 0}, upload=573]: Begin upload segment... Sep 6 14:07:59.090671286 (39m35.130244406s) [seg_pos={0 0}, upload=573]: Done begin upload segment. Sep 6 14:07:59.090694731 (39m35.130267851s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090700253 (39m35.130273373s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090737634 (39m35.130310744s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090785436 (39m35.130358556s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091146606 (39m35.130719727s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091511394 (39m35.131084514s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091781421 (39m35.131354541s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092038142 (39m35.131611262s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092299963 (39m35.131873073s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092543239 (39m35.132116359s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092745115 (39m35.132318235s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092944447 (39m35.132517567s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093145752 (39m35.132718872s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093382265 (39m35.132955385s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093579211 (39m35.133152331s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093764656 (39m35.133337776s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:08:00.937684886 (39m36.977258016s) [upload=573]: Next returned nil segment Sep 6 14:08:00.937711106 (39m36.977284226s) [upload=573]: Waiting for error group managing segments... Sep 6 14:08:01.229627256 (39m37.269200376s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.239225592 (39m37.278798712s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.247313068 (39m37.286886218s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.249756584 (39m37.289329704s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.251873975 (39m37.291447105s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.274644587 (39m37.314217717s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.285486915 (39m37.325060035s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.359873446 (39m37.399446566s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.415375370 (39m37.45494849s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.432187200 (39m37.47176032s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.449428201 (39m37.489001321s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.529655178 (39m37.569228298s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716515189 (39m37.756088309s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716548723 (39m37.756121853s) [seg_pos={0 0}, upload=573]: Segment reached optimal threshold of 13 pieces. Sep 6 14:08:01.716873925 (39m37.756447055s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection Sep 6 14:08:01.716895096 (39m37.756468216s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection Sep 6 14:08:01.716949470 (39m37.75652259s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection Sep 6 14:08:01.716976963 (39m37.756550083s) [seg_pos={0 0}, upload=573]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:08:01.717065301 (39m37.756638421s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:08:01.717069790 (39m37.75664292s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:08:02.058321383 (39m38.097894514s) [upload=573]: Done uploading segments... Sep 6 14:08:02.058345169 (39m38.097918289s) [upload=573]: Upload finished. err: <nil> Duration: 15.225796295s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 20 out of 60 is failed (threshold=3) [.........XX......XX........XXX...XX...XX..XXX..XX..XX.....XX] duration: 30.002034068s FAIL (2025-09-06 14:08:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:07:58.816734823 (39m34.856307943s) [upload=573]: Starting upload Sep 6 14:07:58.816793234 (39m34.856366354s) [upload=573]: Done starting upload Sep 6 14:07:58.816878788 (39m34.856451908s) [upload=573]: Uploading segments... Sep 6 14:07:58.817014688 (39m34.856587808s) [upload=573]: Got next segment. Inline: false Sep 6 14:07:58.817038183 (39m34.856611303s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:07:58.817042611 (39m34.856615731s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:07:59.090619398 (39m35.130192558s) [seg_pos={0 0}, upload=573]: Begin upload segment... Sep 6 14:07:59.090671286 (39m35.130244406s) [seg_pos={0 0}, upload=573]: Done begin upload segment. Sep 6 14:07:59.090694731 (39m35.130267851s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090700253 (39m35.130273373s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090737634 (39m35.130310744s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.090785436 (39m35.130358556s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091146606 (39m35.130719727s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091511394 (39m35.131084514s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.091781421 (39m35.131354541s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092038142 (39m35.131611262s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092299963 (39m35.131873073s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092543239 (39m35.132116359s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092745115 (39m35.132318235s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.092944447 (39m35.132517567s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093145752 (39m35.132718872s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093382265 (39m35.132955385s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093579211 (39m35.133152331s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:07:59.093764656 (39m35.133337776s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Uploading piece... Sep 6 14:08:00.937684886 (39m36.977258016s) [upload=573]: Next returned nil segment Sep 6 14:08:00.937711106 (39m36.977284226s) [upload=573]: Waiting for error group managing segments... Sep 6 14:08:01.229627256 (39m37.269200376s) [noise=true , address=45.91.116.13:4155 , piece_id=75AGPIKAGOEOQHY7OKVSKCHYBGFVYZCYEW3CPMLLA3PXSDLDWIOA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.239225592 (39m37.278798712s) [noise=true , address=45.91.116.14:4189 , piece_id=RSL5SHRTXK5ASAZ3GDGPOM7MTBO4A5WLQIPQMY6OKNBNTIWO45LA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.247313068 (39m37.286886218s) [noise=true , address=45.91.116.11:4118 , piece_id=ILSTLBOASI5JCDVTA5XF2ZVFIGQGKG6XP3CYOCET743AUFJAEI2Q, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.249756584 (39m37.289329704s) [noise=true , address=77.90.31.15:4697 , piece_id=YEJ7UE4R3GLHSF4QTW5PXBUBIN6ZZ4OTWFLVIETFG2LEYUSWEUYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.251873975 (39m37.291447105s) [noise=true , address=45.91.116.7:4010 , piece_id=DCLKXVSYKPPXDYRGYR67WK6CI7HDE7OXHAJWVICTIQAXAE7S3PMA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.274644587 (39m37.314217717s) [noise=true , address=45.91.116.14:4191 , piece_id=7QQASD6N5PKXSTZKNMB7KOV23J4EKWQ6DP7QLEVKYVPZSVFZLRYA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.285486915 (39m37.325060035s) [noise=true , address=45.91.116.4:4937 , piece_id=5CFUJ7QI25ZTSWZXIFZ3TWSENHAAIJMQ3NWZQ44CZ7PVLOL5IIGA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.359873446 (39m37.399446566s) [noise=true , address=77.90.31.14:4661 , piece_id=ZQDU35RUJBF74WKVRY6EH5KHN7URJH3JSWBJTXEZX5NWKBB5BSTQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.415375370 (39m37.45494849s) [noise=true , address=45.91.116.12:4226 , piece_id=4ZVFVSDWFS3A56W7ROVR66QO44R3BBABVWXYCIEASOBLYRTYA3XA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.432187200 (39m37.47176032s) [noise=true , address=77.90.31.12:4590 , piece_id=JM4ALARVLWPJSOZY6NXPCDB2B5UD3T3F5IWS45HRZLJU5IRGNSWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.449428201 (39m37.489001321s) [noise=true , address=45.91.116.8:4047 , piece_id=KSULZ5ZMKZ5IEKN3IR34CR4DZOTKPF2BGC3DHBHSKEAQTFD4MJLA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.529655178 (39m37.569228298s) [noise=true , address=77.90.31.16:4735 , piece_id=2OYPKBTOEJJGZEFLAPJE35I4NMOYQZKIYHHXYHSMBRXG3SHTWJVA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716515189 (39m37.756088309s) [noise=true , address=45.91.116.10:4081 , piece_id=SJ6PVOFBFOCVIWKH6DK53NQBCLXODIOV2IIZXOMKXCWUGACR6ZAA, seg_pos={0 0}, upload=573]: Done uploading piece. err: <nil> Sep 6 14:08:01.716548723 (39m37.756121853s) [seg_pos={0 0}, upload=573]: Segment reached optimal threshold of 13 pieces. Sep 6 14:08:01.716873925 (39m37.756447055s) [noise=true , address=77.90.31.10:4520 , piece_id=JTUPJDIQHMM5BK75P7CSZBNW2GWWLBA7LFXLBRSIFI6RDUTMRYWQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ehpG3YwRKWB8Z4qw9MZofFC4K8XNm16p27HJ2fZzvLYJTgtEp): protocol: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44894->77.90.31.10:4520: use of closed network connection Sep 6 14:08:01.716895096 (39m37.756468216s) [noise=true , address=45.91.116.8:4045 , piece_id=E62YWG7JWJYYLAMCFTTNQ5TZ7KKOFJWMBIEZVRBGEOAP2KLH5QDQ, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9): protocol: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54494->45.91.116.8:4045: use of closed network connection Sep 6 14:08:01.716949470 (39m37.75652259s) [noise=true , address=45.91.116.10:4082 , piece_id=LDLBXW22GWLKOERH3Y2EVN3LQM6GTYPZLZ5SPMKPB24627RTTCFA, seg_pos={0 0}, upload=573]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj): protocol: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36880->45.91.116.10:4082: use of closed network connection Sep 6 14:08:01.716976963 (39m37.756550083s) [seg_pos={0 0}, upload=573]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:08:01.717065301 (39m37.756638421s) [upload=573]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:08:01.717069790 (39m37.75664292s) [upload=573]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:08:02.058321383 (39m38.097894514s) [upload=573]: Done uploading segments... Sep 6 14:08:02.058345169 (39m38.097918289s) [upload=573]: Upload finished. err: <nil> Duration: 30.002046212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 19 out of 60 is failed (threshold=3) [..........XX......XX........XXX...XX...XX..XXX..XX..XX.....X] duration: 30.001934378s FAIL (2025-09-06 14:07:57 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:07:28.566181495 (39m4.605754615s) [upload=562]: Starting upload Sep 6 14:07:28.566236210 (39m4.60580932s) [upload=562]: Done starting upload Sep 6 14:07:28.566316995 (39m4.605890115s) [upload=562]: Uploading segments... Sep 6 14:07:28.566426324 (39m4.605999444s) [upload=562]: Got next segment. Inline: false Sep 6 14:07:28.566456031 (39m4.606029151s) [upload=562]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:07:28.566466110 (39m4.60603923s) [upload=562]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:07:29.401716073 (39m5.441289193s) [seg_pos={0 0}, upload=562]: Begin upload segment... Sep 6 14:07:29.401774695 (39m5.441347815s) [seg_pos={0 0}, upload=562]: Done begin upload segment. Sep 6 14:07:29.401831133 (39m5.441404253s) [noise=true , address=45.91.116.4:4938 , piece_id=DDE43CEBEU4WA4CZB2SWI6F7QQ7DTLFB2F6CBUPDCAFZ4HM7JUUQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.401835441 (39m5.441408551s) [noise=true , address=45.91.116.12:4226 , piece_id=5X7DOWCUGSGPHAWHX3HTAE76BYIB2TH2YPCAKEORTEQU4SHJEKIA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.402333575 (39m5.441906695s) [noise=true , address=45.91.116.8:4047 , piece_id=R35KFAK2K2JIQWDJZ6QKOLXU77VCNHVKZ2WKW5XQN34SXOI7UVJA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.402702871 (39m5.442276031s) [noise=true , address=45.91.116.7:4009 , piece_id=LUVYH2BF5WZ7S2UKVKFUZRM2DIFK3R6HEPOQYGDNDNZP6OYX554A, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.403096425 (39m5.442669545s) [noise=true , address=45.91.116.11:4119 , piece_id=Z3SBCXVRGRI3BJHHRO57BPLMPHYJHSFUQJM4MF75HN2AUPJMRCTQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.403719907 (39m5.443293057s) [noise=true , address=77.90.31.16:4733 , piece_id=UWK6E5BJON43VMOIUXN2MMRO462VKPPGXXT7JK2MK2TWSBYKCTWQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.404362928 (39m5.443936048s) [noise=true , address=77.90.31.16:4734 , piece_id=2UEKPMVI6Z722XWZYIOSI22MWUSHSPK4B22IXVD5PYVDILMDREAQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.404619018 (39m5.444192138s) [noise=true , address=45.91.116.8:4045 , piece_id=JJM3DKWLNUWMQ2RTJ25QDNTTTNBOUBRVVJKRNXHLIIVE5R6YOXVA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.404845051 (39m5.444418171s) [noise=true , address=77.90.31.19:4842 , piece_id=M5AGZYIAOCIUAHBDVCS5JN7LX5FWO2THYVYZUPG4SLYBKVOV5YOQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.405245356 (39m5.444818476s) [noise=true , address=45.91.116.4:4940 , piece_id=ERJLSUF5FEAGAT54LYROY3Q4OQZ5P55Q2TSAJPYBWEZMULVTNFUQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.405498971 (39m5.445072091s) [noise=true , address=45.91.116.5:4297 , piece_id=IO3W6PVLNNZ3DDTL6Z3ZWSEBRM3AJOSIJYG4CYAUKAR6MOXIQ6NQ, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.405775411 (39m5.445348521s) [noise=true , address=77.90.31.19:4843 , piece_id=2J7H6367YWLGKGYYL2NZ3XSPLOSHJE3QBLQFT5AU2HKZHUA5PCEA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.406041800 (39m5.44561491s) [noise=true , address=45.91.116.6:4974 , piece_id=E7CCLC5CFCHX6EUMMYO7ISFAIZ6F3ULUTNSI4NV5FHER4SZIK2SA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.406269727 (39m5.445842847s) [noise=true , address=45.91.116.9:4262 , piece_id=MU5XLIAMUG2XRUFOHVDYCA7FPKPITO5FGOPHOEJP4K3LPUQ6I4JA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.406457807 (39m5.446030927s) [noise=true , address=77.90.31.15:4697 , piece_id=4ANS3BW76XIBBC4R7J73ASLWB227FWIVRT3XK5OH5LBQH7GP2YTA, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:29.406723625 (39m5.446296745s) [noise=true , address=77.90.31.21:4878 , piece_id=WWJNWG7E3BT4WCSYPHSWJO7C7FY7DDICCW6PFX6VRNHX4QJFEU5Q, seg_pos={0 0}, upload=562]: Uploading piece... Sep 6 14:07:31.242880671 (39m7.282453791s) [upload=562]: Next returned nil segment Sep 6 14:07:31.242901289 (39m7.2824744s) [upload=562]: Waiting for error group managing segments... Sep 6 14:07:31.543897254 (39m7.583470374s) [noise=true , address=45.91.116.8:4045 , piece_id=JJM3DKWLNUWMQ2RTJ25QDNTTTNBOUBRVVJKRNXHLIIVE5R6YOXVA, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.584085695 (39m7.623658815s) [noise=true , address=77.90.31.19:4843 , piece_id=2J7H6367YWLGKGYYL2NZ3XSPLOSHJE3QBLQFT5AU2HKZHUA5PCEA, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.607249069 (39m7.646822189s) [noise=true , address=77.90.31.19:4842 , piece_id=M5AGZYIAOCIUAHBDVCS5JN7LX5FWO2THYVYZUPG4SLYBKVOV5YOQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.631838851 (39m7.671411971s) [noise=true , address=77.90.31.15:4697 , piece_id=4ANS3BW76XIBBC4R7J73ASLWB227FWIVRT3XK5OH5LBQH7GP2YTA, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.653241166 (39m7.692814286s) [noise=true , address=45.91.116.8:4047 , piece_id=R35KFAK2K2JIQWDJZ6QKOLXU77VCNHVKZ2WKW5XQN34SXOI7UVJA, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.664318793 (39m7.703891913s) [noise=true , address=77.90.31.16:4734 , piece_id=2UEKPMVI6Z722XWZYIOSI22MWUSHSPK4B22IXVD5PYVDILMDREAQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.704647102 (39m7.744220223s) [noise=true , address=45.91.116.5:4297 , piece_id=IO3W6PVLNNZ3DDTL6Z3ZWSEBRM3AJOSIJYG4CYAUKAR6MOXIQ6NQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.732849969 (39m7.772423109s) [noise=true , address=45.91.116.4:4938 , piece_id=DDE43CEBEU4WA4CZB2SWI6F7QQ7DTLFB2F6CBUPDCAFZ4HM7JUUQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.739276747 (39m7.778849867s) [noise=true , address=45.91.116.12:4226 , piece_id=5X7DOWCUGSGPHAWHX3HTAE76BYIB2TH2YPCAKEORTEQU4SHJEKIA, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.767652756 (39m7.807225876s) [noise=true , address=45.91.116.4:4940 , piece_id=ERJLSUF5FEAGAT54LYROY3Q4OQZ5P55Q2TSAJPYBWEZMULVTNFUQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.769417722 (39m7.808990843s) [noise=true , address=77.90.31.16:4733 , piece_id=UWK6E5BJON43VMOIUXN2MMRO462VKPPGXXT7JK2MK2TWSBYKCTWQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:31.782580811 (39m7.822153921s) [noise=true , address=45.91.116.6:4974 , piece_id=E7CCLC5CFCHX6EUMMYO7ISFAIZ6F3ULUTNSI4NV5FHER4SZIK2SA, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:32.294308746 (39m8.333881866s) [noise=true , address=45.91.116.7:4009 , piece_id=LUVYH2BF5WZ7S2UKVKFUZRM2DIFK3R6HEPOQYGDNDNZP6OYX554A, seg_pos={0 0}, upload=562]: Done uploading piece. err: <nil> Sep 6 14:07:32.294334395 (39m8.333907515s) [seg_pos={0 0}, upload=562]: Segment reached optimal threshold of 13 pieces. Sep 6 14:07:32.294457982 (39m8.334031132s) [noise=true , address=77.90.31.21:4878 , piece_id=WWJNWG7E3BT4WCSYPHSWJO7C7FY7DDICCW6PFX6VRNHX4QJFEU5Q, seg_pos={0 0}, upload=562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 6 14:07:32.294500472 (39m8.334073593s) [noise=true , address=45.91.116.11:4119 , piece_id=Z3SBCXVRGRI3BJHHRO57BPLMPHYJHSFUQJM4MF75HN2AUPJMRCTQ, seg_pos={0 0}, upload=562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: expected piece hash; context canceled; EOF Sep 6 14:07:32.294568774 (39m8.334141894s) [noise=true , address=45.91.116.9:4262 , piece_id=MU5XLIAMUG2XRUFOHVDYCA7FPKPITO5FGOPHOEJP4K3LPUQ6I4JA, seg_pos={0 0}, upload=562]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rebnup7LaVo85uFG24Afr6oReVAc5NiAARKApQaP8soPczPto): protocol: expected piece hash; context canceled; EOF Sep 6 14:07:32.294589973 (39m8.334163094s) [seg_pos={0 0}, upload=562]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:07:32.294645840 (39m8.33421896s) [upload=562]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:07:32.294649909 (39m8.334223029s) [upload=562]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:07:32.642671087 (39m8.682244237s) [upload=562]: Done uploading segments... Sep 6 14:07:32.642763994 (39m8.682337114s) [upload=562]: Upload finished. err: <nil> Duration: 30.001959317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [...........XX......XX........XXX...XX...XX..XXX..XX..XX.....] duration: 21.343148601s FAIL (2025-09-06 14:06:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:02:16.586057763 (33m52.625630883s) [upload=486]: Starting upload Sep 6 14:02:16.586119862 (33m52.625692983s) [upload=486]: Done starting upload Sep 6 14:02:16.586288365 (33m52.625861485s) [upload=486]: Uploading segments... Sep 6 14:02:16.586325866 (33m52.625898986s) [upload=486]: Got next segment. Inline: false Sep 6 14:02:16.586348981 (33m52.625922101s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:02:16.586353058 (33m52.625926188s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:02:16.834414470 (33m52.87398758s) [seg_pos={0 0}, upload=486]: Begin upload segment... Sep 6 14:02:16.834455599 (33m52.87402872s) [seg_pos={0 0}, upload=486]: Done begin upload segment. Sep 6 14:02:16.834478283 (33m52.874051403s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834542395 (33m52.874115515s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834880111 (33m52.874453231s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834968942 (33m52.874542062s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835202048 (33m52.874775158s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835250971 (33m52.874824091s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835569069 (33m52.875142189s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835584229 (33m52.875157349s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835849687 (33m52.875422807s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835900384 (33m52.875473504s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836135003 (33m52.875708123s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836299608 (33m52.875872728s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836358260 (33m52.87593138s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836684123 (33m52.876257243s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837094729 (33m52.876667849s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837388802 (33m52.876961922s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:18.654229966 (33m54.693803086s) [upload=486]: Next returned nil segment Sep 6 14:02:18.654277598 (33m54.693850708s) [upload=486]: Waiting for error group managing segments... Sep 6 14:02:18.966114776 (33m55.005687897s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.038494557 (33m55.078067677s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.043536805 (33m55.083109935s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.046275556 (33m55.085848676s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.062657624 (33m55.102230754s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.065123011 (33m55.104696131s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.081223169 (33m55.120796289s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.083713865 (33m55.123286985s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.106402781 (33m55.145975901s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.113126878 (33m55.152699998s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.145699967 (33m55.185273088s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.150385643 (33m55.189958763s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164906119 (33m55.204479239s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164937048 (33m55.204510168s) [seg_pos={0 0}, upload=486]: Segment reached optimal threshold of 13 pieces. Sep 6 14:02:19.165053982 (33m55.204627102s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165211193 (33m55.204784313s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection Sep 6 14:02:19.165223706 (33m55.204796826s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165274574 (33m55.204847694s) [seg_pos={0 0}, upload=486]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:02:19.165357973 (33m55.204931093s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:02:19.165363123 (33m55.204936243s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:02:19.581846836 (33m55.621419976s) [upload=486]: Done uploading segments... Sep 6 14:02:19.581868798 (33m55.621441928s) [upload=486]: Upload finished. err: <nil> Duration: 21.343174872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [............XX......XX........XXX...XX...XX..XXX..XX..XX....] duration: 12.264772196s FAIL (2025-09-06 14:05:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:02:16.586057763 (33m52.625630883s) [upload=486]: Starting upload Sep 6 14:02:16.586119862 (33m52.625692983s) [upload=486]: Done starting upload Sep 6 14:02:16.586288365 (33m52.625861485s) [upload=486]: Uploading segments... Sep 6 14:02:16.586325866 (33m52.625898986s) [upload=486]: Got next segment. Inline: false Sep 6 14:02:16.586348981 (33m52.625922101s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:02:16.586353058 (33m52.625926188s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:02:16.834414470 (33m52.87398758s) [seg_pos={0 0}, upload=486]: Begin upload segment... Sep 6 14:02:16.834455599 (33m52.87402872s) [seg_pos={0 0}, upload=486]: Done begin upload segment. Sep 6 14:02:16.834478283 (33m52.874051403s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834542395 (33m52.874115515s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834880111 (33m52.874453231s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834968942 (33m52.874542062s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835202048 (33m52.874775158s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835250971 (33m52.874824091s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835569069 (33m52.875142189s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835584229 (33m52.875157349s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835849687 (33m52.875422807s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835900384 (33m52.875473504s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836135003 (33m52.875708123s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836299608 (33m52.875872728s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836358260 (33m52.87593138s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836684123 (33m52.876257243s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837094729 (33m52.876667849s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837388802 (33m52.876961922s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:18.654229966 (33m54.693803086s) [upload=486]: Next returned nil segment Sep 6 14:02:18.654277598 (33m54.693850708s) [upload=486]: Waiting for error group managing segments... Sep 6 14:02:18.966114776 (33m55.005687897s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.038494557 (33m55.078067677s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.043536805 (33m55.083109935s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.046275556 (33m55.085848676s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.062657624 (33m55.102230754s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.065123011 (33m55.104696131s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.081223169 (33m55.120796289s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.083713865 (33m55.123286985s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.106402781 (33m55.145975901s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.113126878 (33m55.152699998s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.145699967 (33m55.185273088s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.150385643 (33m55.189958763s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164906119 (33m55.204479239s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164937048 (33m55.204510168s) [seg_pos={0 0}, upload=486]: Segment reached optimal threshold of 13 pieces. Sep 6 14:02:19.165053982 (33m55.204627102s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165211193 (33m55.204784313s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection Sep 6 14:02:19.165223706 (33m55.204796826s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165274574 (33m55.204847694s) [seg_pos={0 0}, upload=486]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:02:19.165357973 (33m55.204931093s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:02:19.165363123 (33m55.204936243s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:02:19.581846836 (33m55.621419976s) [upload=486]: Done uploading segments... Sep 6 14:02:19.581868798 (33m55.621441928s) [upload=486]: Upload finished. err: <nil> Duration: 12.264799671s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [.............XX......XX........XXX...XX...XX..XXX..XX..XX...] duration: 8.572964023s FAIL (2025-09-06 14:04:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:02:16.586057763 (33m52.625630883s) [upload=486]: Starting upload Sep 6 14:02:16.586119862 (33m52.625692983s) [upload=486]: Done starting upload Sep 6 14:02:16.586288365 (33m52.625861485s) [upload=486]: Uploading segments... Sep 6 14:02:16.586325866 (33m52.625898986s) [upload=486]: Got next segment. Inline: false Sep 6 14:02:16.586348981 (33m52.625922101s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:02:16.586353058 (33m52.625926188s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:02:16.834414470 (33m52.87398758s) [seg_pos={0 0}, upload=486]: Begin upload segment... Sep 6 14:02:16.834455599 (33m52.87402872s) [seg_pos={0 0}, upload=486]: Done begin upload segment. Sep 6 14:02:16.834478283 (33m52.874051403s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834542395 (33m52.874115515s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834880111 (33m52.874453231s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834968942 (33m52.874542062s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835202048 (33m52.874775158s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835250971 (33m52.874824091s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835569069 (33m52.875142189s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835584229 (33m52.875157349s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835849687 (33m52.875422807s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835900384 (33m52.875473504s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836135003 (33m52.875708123s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836299608 (33m52.875872728s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836358260 (33m52.87593138s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836684123 (33m52.876257243s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837094729 (33m52.876667849s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837388802 (33m52.876961922s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:18.654229966 (33m54.693803086s) [upload=486]: Next returned nil segment Sep 6 14:02:18.654277598 (33m54.693850708s) [upload=486]: Waiting for error group managing segments... Sep 6 14:02:18.966114776 (33m55.005687897s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.038494557 (33m55.078067677s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.043536805 (33m55.083109935s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.046275556 (33m55.085848676s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.062657624 (33m55.102230754s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.065123011 (33m55.104696131s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.081223169 (33m55.120796289s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.083713865 (33m55.123286985s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.106402781 (33m55.145975901s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.113126878 (33m55.152699998s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.145699967 (33m55.185273088s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.150385643 (33m55.189958763s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164906119 (33m55.204479239s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164937048 (33m55.204510168s) [seg_pos={0 0}, upload=486]: Segment reached optimal threshold of 13 pieces. Sep 6 14:02:19.165053982 (33m55.204627102s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165211193 (33m55.204784313s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection Sep 6 14:02:19.165223706 (33m55.204796826s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165274574 (33m55.204847694s) [seg_pos={0 0}, upload=486]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:02:19.165357973 (33m55.204931093s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:02:19.165363123 (33m55.204936243s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:02:19.581846836 (33m55.621419976s) [upload=486]: Done uploading segments... Sep 6 14:02:19.581868798 (33m55.621441928s) [upload=486]: Upload finished. err: <nil> Duration: 8.572987808s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [..............XX......XX........XXX...XX...XX..XXX..XX..XX..] duration: 15.174071759s FAIL (2025-09-06 14:04:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:02:16.586057763 (33m52.625630883s) [upload=486]: Starting upload Sep 6 14:02:16.586119862 (33m52.625692983s) [upload=486]: Done starting upload Sep 6 14:02:16.586288365 (33m52.625861485s) [upload=486]: Uploading segments... Sep 6 14:02:16.586325866 (33m52.625898986s) [upload=486]: Got next segment. Inline: false Sep 6 14:02:16.586348981 (33m52.625922101s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:02:16.586353058 (33m52.625926188s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:02:16.834414470 (33m52.87398758s) [seg_pos={0 0}, upload=486]: Begin upload segment... Sep 6 14:02:16.834455599 (33m52.87402872s) [seg_pos={0 0}, upload=486]: Done begin upload segment. Sep 6 14:02:16.834478283 (33m52.874051403s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834542395 (33m52.874115515s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834880111 (33m52.874453231s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834968942 (33m52.874542062s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835202048 (33m52.874775158s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835250971 (33m52.874824091s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835569069 (33m52.875142189s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835584229 (33m52.875157349s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835849687 (33m52.875422807s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835900384 (33m52.875473504s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836135003 (33m52.875708123s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836299608 (33m52.875872728s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836358260 (33m52.87593138s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836684123 (33m52.876257243s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837094729 (33m52.876667849s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837388802 (33m52.876961922s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:18.654229966 (33m54.693803086s) [upload=486]: Next returned nil segment Sep 6 14:02:18.654277598 (33m54.693850708s) [upload=486]: Waiting for error group managing segments... Sep 6 14:02:18.966114776 (33m55.005687897s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.038494557 (33m55.078067677s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.043536805 (33m55.083109935s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.046275556 (33m55.085848676s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.062657624 (33m55.102230754s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.065123011 (33m55.104696131s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.081223169 (33m55.120796289s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.083713865 (33m55.123286985s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.106402781 (33m55.145975901s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.113126878 (33m55.152699998s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.145699967 (33m55.185273088s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.150385643 (33m55.189958763s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164906119 (33m55.204479239s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164937048 (33m55.204510168s) [seg_pos={0 0}, upload=486]: Segment reached optimal threshold of 13 pieces. Sep 6 14:02:19.165053982 (33m55.204627102s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165211193 (33m55.204784313s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection Sep 6 14:02:19.165223706 (33m55.204796826s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165274574 (33m55.204847694s) [seg_pos={0 0}, upload=486]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:02:19.165357973 (33m55.204931093s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:02:19.165363123 (33m55.204936243s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:02:19.581846836 (33m55.621419976s) [upload=486]: Done uploading segments... Sep 6 14:02:19.581868798 (33m55.621441928s) [upload=486]: Upload finished. err: <nil> Duration: 15.174101345s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [...............XX......XX........XXX...XX...XX..XXX..XX..XX.] duration: 25.704754328s FAIL (2025-09-06 14:03:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:02:16.586057763 (33m52.625630883s) [upload=486]: Starting upload Sep 6 14:02:16.586119862 (33m52.625692983s) [upload=486]: Done starting upload Sep 6 14:02:16.586288365 (33m52.625861485s) [upload=486]: Uploading segments... Sep 6 14:02:16.586325866 (33m52.625898986s) [upload=486]: Got next segment. Inline: false Sep 6 14:02:16.586348981 (33m52.625922101s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:02:16.586353058 (33m52.625926188s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:02:16.834414470 (33m52.87398758s) [seg_pos={0 0}, upload=486]: Begin upload segment... Sep 6 14:02:16.834455599 (33m52.87402872s) [seg_pos={0 0}, upload=486]: Done begin upload segment. Sep 6 14:02:16.834478283 (33m52.874051403s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834542395 (33m52.874115515s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834880111 (33m52.874453231s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834968942 (33m52.874542062s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835202048 (33m52.874775158s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835250971 (33m52.874824091s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835569069 (33m52.875142189s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835584229 (33m52.875157349s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835849687 (33m52.875422807s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835900384 (33m52.875473504s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836135003 (33m52.875708123s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836299608 (33m52.875872728s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836358260 (33m52.87593138s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836684123 (33m52.876257243s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837094729 (33m52.876667849s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837388802 (33m52.876961922s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:18.654229966 (33m54.693803086s) [upload=486]: Next returned nil segment Sep 6 14:02:18.654277598 (33m54.693850708s) [upload=486]: Waiting for error group managing segments... Sep 6 14:02:18.966114776 (33m55.005687897s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.038494557 (33m55.078067677s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.043536805 (33m55.083109935s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.046275556 (33m55.085848676s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.062657624 (33m55.102230754s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.065123011 (33m55.104696131s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.081223169 (33m55.120796289s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.083713865 (33m55.123286985s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.106402781 (33m55.145975901s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.113126878 (33m55.152699998s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.145699967 (33m55.185273088s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.150385643 (33m55.189958763s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164906119 (33m55.204479239s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164937048 (33m55.204510168s) [seg_pos={0 0}, upload=486]: Segment reached optimal threshold of 13 pieces. Sep 6 14:02:19.165053982 (33m55.204627102s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165211193 (33m55.204784313s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection Sep 6 14:02:19.165223706 (33m55.204796826s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165274574 (33m55.204847694s) [seg_pos={0 0}, upload=486]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:02:19.165357973 (33m55.204931093s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:02:19.165363123 (33m55.204936243s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:02:19.581846836 (33m55.621419976s) [upload=486]: Done uploading segments... Sep 6 14:02:19.581868798 (33m55.621441928s) [upload=486]: Upload finished. err: <nil> Duration: 25.70478615s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 18 out of 60 is failed (threshold=3) [................XX......XX........XXX...XX...XX..XXX..XX..XX] duration: 30.002101725s FAIL (2025-09-06 14:02:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:02:16.586057763 (33m52.625630883s) [upload=486]: Starting upload Sep 6 14:02:16.586119862 (33m52.625692983s) [upload=486]: Done starting upload Sep 6 14:02:16.586288365 (33m52.625861485s) [upload=486]: Uploading segments... Sep 6 14:02:16.586325866 (33m52.625898986s) [upload=486]: Got next segment. Inline: false Sep 6 14:02:16.586348981 (33m52.625922101s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:02:16.586353058 (33m52.625926188s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:02:16.834414470 (33m52.87398758s) [seg_pos={0 0}, upload=486]: Begin upload segment... Sep 6 14:02:16.834455599 (33m52.87402872s) [seg_pos={0 0}, upload=486]: Done begin upload segment. Sep 6 14:02:16.834478283 (33m52.874051403s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834542395 (33m52.874115515s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834880111 (33m52.874453231s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.834968942 (33m52.874542062s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835202048 (33m52.874775158s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835250971 (33m52.874824091s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835569069 (33m52.875142189s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835584229 (33m52.875157349s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835849687 (33m52.875422807s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.835900384 (33m52.875473504s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836135003 (33m52.875708123s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836299608 (33m52.875872728s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836358260 (33m52.87593138s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.836684123 (33m52.876257243s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837094729 (33m52.876667849s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:16.837388802 (33m52.876961922s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Uploading piece... Sep 6 14:02:18.654229966 (33m54.693803086s) [upload=486]: Next returned nil segment Sep 6 14:02:18.654277598 (33m54.693850708s) [upload=486]: Waiting for error group managing segments... Sep 6 14:02:18.966114776 (33m55.005687897s) [noise=true , address=45.91.116.7:4009 , piece_id=V32RG4PSIVO7UDDNQODIVSRLTCYCZHP6BKCT2HDPOLPKAQEIO5XA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.038494557 (33m55.078067677s) [noise=true , address=45.91.116.13:4153 , piece_id=G533BKJ3S2KEOTAUG3SSS273QNL6IQVESJUICG3FZSB4ZAOB6MKA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.043536805 (33m55.083109935s) [noise=true , address=45.91.116.5:4299 , piece_id=JNKYJSFNRKZRWBHLL5O5RW627S7BIVQ6JETSB2XRVYHNLFHG6W3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.046275556 (33m55.085848676s) [noise=true , address=45.91.116.12:4226 , piece_id=RIS7B22QHLKUNSCWGR5XC6AFQCO3ME32HXWHLNMBFN4U3U2HWF3A, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.062657624 (33m55.102230754s) [noise=true , address=77.90.31.14:4661 , piece_id=37O6R32KKM7BK4GQDTB4Q2M4DL7OK4V4DH4MWEQAR7HHVEBYO5YQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.065123011 (33m55.104696131s) [noise=true , address=77.90.31.18:4806 , piece_id=B6AFMBG74ZGEQA24RGKUIB6Q4MQ4LRFMUS7O6X3U5IZ7ZCNPB3EQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.081223169 (33m55.120796289s) [noise=true , address=77.90.31.19:4843 , piece_id=I2LPCYG6ICVCDEVPLVZUNW7IXJBPJWEIPIEL2DXSSWKD637UAEVQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.083713865 (33m55.123286985s) [noise=true , address=45.91.116.5:4297 , piece_id=QZBNMRBDS7MJ4LZPX3REWCW6KF3S5ZLOXK2GN3YSGGX6BNTF7CWQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.106402781 (33m55.145975901s) [noise=true , address=45.91.116.8:4047 , piece_id=2W33VQVVCTC4AWMPNEQMRJYFD6TKEHEQWLMMU6O3LR3GEVFAZKCQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.113126878 (33m55.152699998s) [noise=true , address=45.91.116.13:4154 , piece_id=2LAOZWFWQSW2VDTCJMXOORRT5ZN5N3AJ4CIFJI4BNKXLHXUBCVIQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.145699967 (33m55.185273088s) [noise=true , address=45.91.116.6:4973 , piece_id=BNJJUUF2ELRBRYNFGSZ5YSIKVF7E7OWM3YIIOWXOQXC67GFJ4QGA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.150385643 (33m55.189958763s) [noise=true , address=77.90.31.12:4590 , piece_id=U4B2AEEGXLLAQOSSQHUVEGHQXXLMVNTHKJJYWHH5FBXF33SPMLTQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164906119 (33m55.204479239s) [noise=true , address=45.91.116.14:4190 , piece_id=WJR6W2II76CVQKLU2XCFPCMTEI7DFIGF5Z734HZC7TIRNRTDVHJA, seg_pos={0 0}, upload=486]: Done uploading piece. err: <nil> Sep 6 14:02:19.164937048 (33m55.204510168s) [seg_pos={0 0}, upload=486]: Segment reached optimal threshold of 13 pieces. Sep 6 14:02:19.165053982 (33m55.204627102s) [noise=true , address=45.91.116.12:4225 , piece_id=ZICXWEIJ326SFV2X5TV5JDU2FPBGV5CSK6YEDIDSTDTM5PUBGABQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AXaXCJFwDBoz9beNXsHYyfdgRkcX684iB6msjy9d5FkmwCWhy): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165211193 (33m55.204784313s) [noise=true , address=45.91.116.11:4119 , piece_id=EZQTSTPSHPE73276YCNVBVCOU5QHWBKJFUTVAGVPB3RINHOHY7UQ, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42670->45.91.116.11:4119: use of closed network connection Sep 6 14:02:19.165223706 (33m55.204796826s) [noise=true , address=45.91.116.14:4191 , piece_id=ELCEWDNVYZPHTAMFUIHPSGAB7FPX7NQDTXMQKXL2VJF36YCMVG3Q, seg_pos={0 0}, upload=486]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq): protocol: expected piece hash; context canceled; EOF Sep 6 14:02:19.165274574 (33m55.204847694s) [seg_pos={0 0}, upload=486]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:02:19.165357973 (33m55.204931093s) [upload=486]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:02:19.165363123 (33m55.204936243s) [upload=486]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:02:19.581846836 (33m55.621419976s) [upload=486]: Done uploading segments... Sep 6 14:02:19.581868798 (33m55.621441928s) [upload=486]: Upload finished. err: <nil> Duration: 30.002124248s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 17 out of 60 is failed (threshold=3) [.................XX......XX........XXX...XX...XX..XXX..XX..X] duration: 30.001884099s FAIL (2025-09-06 14:02:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 14:01:33.741750142 (33m9.781323262s) [upload=477]: Starting upload Sep 6 14:01:33.741889278 (33m9.781462409s) [upload=477]: Done starting upload Sep 6 14:01:33.742068100 (33m9.78164122s) [upload=477]: Uploading segments... Sep 6 14:01:33.742596021 (33m9.782169141s) [upload=477]: Got next segment. Inline: false Sep 6 14:01:33.742632911 (33m9.782206031s) [upload=477]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 14:01:33.742638001 (33m9.782211111s) [upload=477]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 14:01:33.998270130 (33m10.03784329s) [seg_pos={0 0}, upload=477]: Begin upload segment... Sep 6 14:01:33.998338922 (33m10.037912042s) [seg_pos={0 0}, upload=477]: Done begin upload segment. Sep 6 14:01:33.998370312 (33m10.037943432s) [noise=true , address=45.91.116.8:4045 , piece_id=Y6HK5KMSMEMP333YHSJ7PW42H6AYOT2IBZM42HEWUO3VD3S3W4LQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.998399417 (33m10.037972537s) [noise=true , address=45.91.116.10:4082 , piece_id=DDTRCTIHXQD5PNITQWYM45Z4QRSQO4FNABJ26RJVWTYQI7LL7XJA, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.998813910 (33m10.03838703s) [noise=true , address=77.90.31.15:4697 , piece_id=DCCYLOMHY63EZTEFLTHLRX2FHEHFPMAORIPD5JVRNXCKVG4T7C2A, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.999103906 (33m10.038677026s) [noise=true , address=45.91.116.9:4261 , piece_id=YI45TK7WCGDZAFAPE4D4QXRZ6PFSUNVZIGF3MTH5UEASK4EWPEIQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.999191243 (33m10.038764363s) [noise=true , address=45.91.116.12:4227 , piece_id=XRZX43SNUDNBGRWPCCULRQZDA6YMEMDKVNQEJLYKQR74GHZRFPQA, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.999561221 (33m10.039134341s) [noise=true , address=45.91.116.11:4119 , piece_id=X427Q5E2NP4SMQA776E6SZJPC6HQCAVRWCJWOHLJIORDWH5O3PFQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.999747588 (33m10.039320728s) [noise=true , address=77.90.31.15:4698 , piece_id=EEXKIIU5KKXNJ26I5CJQKQUH3IUII7JFMQXIW2HWGOK56MPO3NMQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:33.999843391 (33m10.039416511s) [noise=true , address=45.91.116.7:4009 , piece_id=3EBJZZMFXHIFL4FYIMRGAQWMXFUWPBKXT2MV75Y5EX7KEHIBT2RA, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.000094771 (33m10.039667901s) [noise=true , address=77.90.31.21:4877 , piece_id=LXJ7JWMNDYQ6RXDDBCLSWZWSX4LI5FMHFCYY26X6DFFFM7U2R6GQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.000372152 (33m10.039945282s) [noise=true , address=45.91.116.5:4298 , piece_id=BKYAR4W6HGR52FRFJ3IPI44SIOSVEXKWXNXNYCD3CPIURVA7ZE2A, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.001001496 (33m10.040574646s) [noise=true , address=45.91.116.6:4975 , piece_id=WY22URMYXYLQ3G4LAGMVY2TRDQZMGM22VF4OUURLWR22TUAP5VEQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.001533124 (33m10.041106254s) [noise=true , address=45.91.116.4:4937 , piece_id=RT4B6ZDMBARTHSVXGYQWZJVIBT4NXACVR4PWDWKYG4X2K65G27KA, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.002098526 (33m10.041671777s) [noise=true , address=77.90.31.17:4772 , piece_id=KDDX5XBJLECNONUC3WIXEXCRRAF744LJXPLYNZ57MFTRYRYO63TQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.011953724 (33m10.051526844s) [noise=true , address=77.90.31.19:4841 , piece_id=EXIOVCRZVVGH544BKHFM2QMAAKSWUJPDBTQR24JDLSRQYJNT3MFQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.015010022 (33m10.054583142s) [noise=true , address=45.91.116.9:4262 , piece_id=PCTTZLIIHADS353BOIW4SBVBVQR2OR3GBDRNECNUO2MY7BPFVWXQ, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:34.019103165 (33m10.058676285s) [noise=true , address=77.90.31.18:4806 , piece_id=XGD22KBAQJ6GGN2WC6TJADBZ3GRT3QWZBU2TXQFSQXYNJHCA4IOA, seg_pos={0 0}, upload=477]: Uploading piece... Sep 6 14:01:35.852623725 (33m11.892196845s) [upload=477]: Next returned nil segment Sep 6 14:01:35.852639416 (33m11.892212536s) [upload=477]: Waiting for error group managing segments... Sep 6 14:01:36.178289706 (33m12.217862866s) [noise=true , address=45.91.116.10:4082 , piece_id=DDTRCTIHXQD5PNITQWYM45Z4QRSQO4FNABJ26RJVWTYQI7LL7XJA, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.188273471 (33m12.227846591s) [noise=true , address=45.91.116.6:4975 , piece_id=WY22URMYXYLQ3G4LAGMVY2TRDQZMGM22VF4OUURLWR22TUAP5VEQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.199662423 (33m12.239235543s) [noise=true , address=45.91.116.4:4937 , piece_id=RT4B6ZDMBARTHSVXGYQWZJVIBT4NXACVR4PWDWKYG4X2K65G27KA, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.203415345 (33m12.242988485s) [noise=true , address=45.91.116.9:4261 , piece_id=YI45TK7WCGDZAFAPE4D4QXRZ6PFSUNVZIGF3MTH5UEASK4EWPEIQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.205790981 (33m12.245364091s) [noise=true , address=45.91.116.8:4045 , piece_id=Y6HK5KMSMEMP333YHSJ7PW42H6AYOT2IBZM42HEWUO3VD3S3W4LQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.223717103 (33m12.263290233s) [noise=true , address=77.90.31.15:4698 , piece_id=EEXKIIU5KKXNJ26I5CJQKQUH3IUII7JFMQXIW2HWGOK56MPO3NMQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.227140764 (33m12.266713894s) [noise=true , address=45.91.116.9:4262 , piece_id=PCTTZLIIHADS353BOIW4SBVBVQR2OR3GBDRNECNUO2MY7BPFVWXQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.271698326 (33m12.311271446s) [noise=true , address=45.91.116.7:4009 , piece_id=3EBJZZMFXHIFL4FYIMRGAQWMXFUWPBKXT2MV75Y5EX7KEHIBT2RA, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.277337467 (33m12.316910587s) [noise=true , address=45.91.116.5:4298 , piece_id=BKYAR4W6HGR52FRFJ3IPI44SIOSVEXKWXNXNYCD3CPIURVA7ZE2A, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.284812200 (33m12.32438532s) [noise=true , address=45.91.116.11:4119 , piece_id=X427Q5E2NP4SMQA776E6SZJPC6HQCAVRWCJWOHLJIORDWH5O3PFQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.320608297 (33m12.360181437s) [noise=true , address=77.90.31.15:4697 , piece_id=DCCYLOMHY63EZTEFLTHLRX2FHEHFPMAORIPD5JVRNXCKVG4T7C2A, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:36.343992643 (33m12.383565763s) [noise=true , address=45.91.116.12:4227 , piece_id=XRZX43SNUDNBGRWPCCULRQZDA6YMEMDKVNQEJLYKQR74GHZRFPQA, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:37.017563909 (33m13.057137029s) [noise=true , address=77.90.31.17:4772 , piece_id=KDDX5XBJLECNONUC3WIXEXCRRAF744LJXPLYNZ57MFTRYRYO63TQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: <nil> Sep 6 14:01:37.017608946 (33m13.057182066s) [seg_pos={0 0}, upload=477]: Segment reached optimal threshold of 13 pieces. Sep 6 14:01:37.017722022 (33m13.057295142s) [noise=true , address=77.90.31.18:4806 , piece_id=XGD22KBAQJ6GGN2WC6TJADBZ3GRT3QWZBU2TXQFSQXYNJHCA4IOA, seg_pos={0 0}, upload=477]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: expected piece hash; context canceled; EOF Sep 6 14:01:37.017881848 (33m13.057454978s) [noise=true , address=77.90.31.21:4877 , piece_id=LXJ7JWMNDYQ6RXDDBCLSWZWSX4LI5FMHFCYY26X6DFFFM7U2R6GQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2): protocol: expected piece hash; context canceled; EOF Sep 6 14:01:37.017925932 (33m13.057499052s) [noise=true , address=77.90.31.19:4841 , piece_id=EXIOVCRZVVGH544BKHFM2QMAAKSWUJPDBTQR24JDLSRQYJNT3MFQ, seg_pos={0 0}, upload=477]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 14:01:37.017944077 (33m13.057517197s) [seg_pos={0 0}, upload=477]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 14:01:37.018028739 (33m13.057601859s) [upload=477]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 14:01:37.018033548 (33m13.057606668s) [upload=477]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 14:01:37.331846558 (33m13.371419678s) [upload=477]: Done uploading segments... Sep 6 14:01:37.331863530 (33m13.37143665s) [upload=477]: Upload finished. err: <nil> Duration: 30.001949023s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [..................XX......XX........XXX...XX...XX..XXX..XX..] duration: 10.038614298s FAIL (2025-09-06 14:00:56 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:59:28.826951007 (31m4.866524127s) [upload=445]: Starting upload Sep 6 13:59:28.827047741 (31m4.866620881s) [upload=445]: Done starting upload Sep 6 13:59:28.827275718 (31m4.866848848s) [upload=445]: Uploading segments... Sep 6 13:59:28.827389937 (31m4.866963057s) [upload=445]: Got next segment. Inline: false Sep 6 13:59:28.827427438 (31m4.867000568s) [upload=445]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:59:28.827446144 (31m4.867019264s) [upload=445]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:59:29.176243838 (31m5.215816958s) [seg_pos={0 0}, upload=445]: Begin upload segment... Sep 6 13:59:29.176289976 (31m5.215863096s) [seg_pos={0 0}, upload=445]: Done begin upload segment. Sep 6 13:59:29.176311858 (31m5.215884978s) [noise=true , address=45.91.116.10:4082 , piece_id=XBRBJBJFXVUJQETVTXTV6ZTULWTM2WWAJIQDDPHHVDFVZ5KDEQAQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176392132 (31m5.215965252s) [noise=true , address=45.91.116.13:4155 , piece_id=EBZOXPRXUFYQ5A62EMLSKT5VHJ2HGHRCNXMETP6LED4COFPUZVXA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176733806 (31m5.216306926s) [noise=true , address=45.91.116.11:4119 , piece_id=6ZF5PBUSVQ622T27KYYVWAHJCJUCCLHHGSK3YQPMOMPL6WEJFLFQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176903921 (31m5.216477041s) [noise=true , address=45.91.116.4:4938 , piece_id=MA4DUBDNGDBN2QDPJNRSHZNOMLELAVYJOJTLSSA2GTEWDZU2HIVA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177129533 (31m5.216702653s) [noise=true , address=45.91.116.8:4045 , piece_id=H2QQ2QDAP6GTKITG7CRR7SDCURHZ4OSIIYFNFAX6KMALRMT2WN5A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177450557 (31m5.217023667s) [noise=true , address=45.91.116.6:4973 , piece_id=JQWN3PO5VQ4CWK3BOFEXMPXWGJWFSM4TCBJWR6DAMNLQYOAYP3AQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177754920 (31m5.21732804s) [noise=true , address=45.91.116.14:4189 , piece_id=2LI5UK7J6S66I2ISJEFLLZ4JWLRXNH4QZAHICOH25Z7YAVBQZ2JA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177779867 (31m5.217352987s) [noise=true , address=45.91.116.10:4081 , piece_id=2CVTVPEZIYIP4QUPRKNQ27MYWVTZUESLE466EDUGPLFM6V32W7DA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178107094 (31m5.217680224s) [noise=true , address=45.91.116.12:4227 , piece_id=45X4BYU4ARSF65AYRRUDOGIE3YCHZL6XV2UG76WMVV6KLSMRYOUQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178122282 (31m5.217695402s) [noise=true , address=77.90.31.13:4627 , piece_id=QSHHL4FUS4B3HITSAQRLDFWRQ35QMRKJOM5NPLQ4KAPYUODK7GRA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178156478 (31m5.217729598s) [noise=true , address=45.91.116.8:4048 , piece_id=COHGPSHEZMEQRTS5KM3UXGLEOGDPBJIIPJTNA54II7L3HRFJBRMQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178409793 (31m5.217982913s) [noise=true , address=45.91.116.6:4975 , piece_id=6E3HA6RCAZAACOOQ6RFZB73C4VAFPVNN4Z2P7CGE4ND2IJEKONUA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.183699795 (31m5.223272905s) [noise=true , address=77.90.31.15:4697 , piece_id=5Q7YYNYI5NYN5HGKJ74RHAYZMAAAWJYEHYXX2VKD7NWF3FNB7J4A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.183980152 (31m5.223553272s) [noise=true , address=45.91.116.7:4009 , piece_id=3ZJHMECXUCBY4ZG7KYFNK4AFT4SKZ2SCFI3XCCUHWGNIRSFA6A6A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.184265027 (31m5.223838147s) [noise=true , address=45.91.116.13:4154 , piece_id=2BYNORNSHZBRAOUU2YBWCBOWGJHXAJT6CXVUPNLZRJGIH2TCJ26A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.184401618 (31m5.223974738s) [noise=true , address=45.91.116.8:4047 , piece_id=YRFF7ABNSHO4G2VUT7LYAZZ3XX7LDORBNL76TUZBWXOYUR43RNYA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:30.987894621 (31m7.027467741s) [upload=445]: Next returned nil segment Sep 6 13:59:30.987918496 (31m7.027491667s) [upload=445]: Waiting for error group managing segments... Sep 6 13:59:31.295099475 (31m7.334672595s) [noise=true , address=45.91.116.8:4045 , piece_id=H2QQ2QDAP6GTKITG7CRR7SDCURHZ4OSIIYFNFAX6KMALRMT2WN5A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.334237326 (31m7.373810446s) [noise=true , address=45.91.116.4:4938 , piece_id=MA4DUBDNGDBN2QDPJNRSHZNOMLELAVYJOJTLSSA2GTEWDZU2HIVA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.351472346 (31m7.391045466s) [noise=true , address=45.91.116.6:4973 , piece_id=JQWN3PO5VQ4CWK3BOFEXMPXWGJWFSM4TCBJWR6DAMNLQYOAYP3AQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.357682198 (31m7.397255318s) [noise=true , address=45.91.116.11:4119 , piece_id=6ZF5PBUSVQ622T27KYYVWAHJCJUCCLHHGSK3YQPMOMPL6WEJFLFQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.376896446 (31m7.416469566s) [noise=true , address=45.91.116.14:4189 , piece_id=2LI5UK7J6S66I2ISJEFLLZ4JWLRXNH4QZAHICOH25Z7YAVBQZ2JA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.379670153 (31m7.419243273s) [noise=true , address=45.91.116.7:4009 , piece_id=3ZJHMECXUCBY4ZG7KYFNK4AFT4SKZ2SCFI3XCCUHWGNIRSFA6A6A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.402121034 (31m7.441694164s) [noise=true , address=45.91.116.13:4154 , piece_id=2BYNORNSHZBRAOUU2YBWCBOWGJHXAJT6CXVUPNLZRJGIH2TCJ26A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.413862341 (31m7.453435451s) [noise=true , address=45.91.116.6:4975 , piece_id=6E3HA6RCAZAACOOQ6RFZB73C4VAFPVNN4Z2P7CGE4ND2IJEKONUA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.415618231 (31m7.455191351s) [noise=true , address=45.91.116.8:4047 , piece_id=YRFF7ABNSHO4G2VUT7LYAZZ3XX7LDORBNL76TUZBWXOYUR43RNYA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.453979285 (31m7.493552405s) [noise=true , address=45.91.116.12:4227 , piece_id=45X4BYU4ARSF65AYRRUDOGIE3YCHZL6XV2UG76WMVV6KLSMRYOUQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.458707914 (31m7.498281034s) [noise=true , address=45.91.116.10:4081 , piece_id=2CVTVPEZIYIP4QUPRKNQ27MYWVTZUESLE466EDUGPLFM6V32W7DA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.495016151 (31m7.534589271s) [noise=true , address=77.90.31.13:4627 , piece_id=QSHHL4FUS4B3HITSAQRLDFWRQ35QMRKJOM5NPLQ4KAPYUODK7GRA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.547548343 (31m7.587121463s) [noise=true , address=45.91.116.10:4082 , piece_id=XBRBJBJFXVUJQETVTXTV6ZTULWTM2WWAJIQDDPHHVDFVZ5KDEQAQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.547579943 (31m7.587153063s) [seg_pos={0 0}, upload=445]: Segment reached optimal threshold of 13 pieces. Sep 6 13:59:31.547726283 (31m7.587299403s) [noise=true , address=77.90.31.15:4697 , piece_id=5Q7YYNYI5NYN5HGKJ74RHAYZMAAAWJYEHYXX2VKD7NWF3FNB7J4A, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.547891519 (31m7.587464639s) [noise=true , address=45.91.116.13:4155 , piece_id=EBZOXPRXUFYQ5A62EMLSKT5VHJ2HGHRCNXMETP6LED4COFPUZVXA, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.547945783 (31m7.587518903s) [noise=true , address=45.91.116.8:4048 , piece_id=COHGPSHEZMEQRTS5KM3UXGLEOGDPBJIIPJTNA54II7L3HRFJBRMQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.548004926 (31m7.587578046s) [seg_pos={0 0}, upload=445]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:59:31.548083657 (31m7.587656777s) [upload=445]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:59:31.548089287 (31m7.587662407s) [upload=445]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:59:31.891122130 (31m7.93069525s) [upload=445]: Done uploading segments... Sep 6 13:59:31.891143722 (31m7.930716842s) [upload=445]: Upload finished. err: <nil> Duration: 10.038642061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [...................XX......XX........XXX...XX...XX..XXX..XX.] duration: 18.18405159s FAIL (2025-09-06 14:00:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:59:28.826951007 (31m4.866524127s) [upload=445]: Starting upload Sep 6 13:59:28.827047741 (31m4.866620881s) [upload=445]: Done starting upload Sep 6 13:59:28.827275718 (31m4.866848848s) [upload=445]: Uploading segments... Sep 6 13:59:28.827389937 (31m4.866963057s) [upload=445]: Got next segment. Inline: false Sep 6 13:59:28.827427438 (31m4.867000568s) [upload=445]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:59:28.827446144 (31m4.867019264s) [upload=445]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:59:29.176243838 (31m5.215816958s) [seg_pos={0 0}, upload=445]: Begin upload segment... Sep 6 13:59:29.176289976 (31m5.215863096s) [seg_pos={0 0}, upload=445]: Done begin upload segment. Sep 6 13:59:29.176311858 (31m5.215884978s) [noise=true , address=45.91.116.10:4082 , piece_id=XBRBJBJFXVUJQETVTXTV6ZTULWTM2WWAJIQDDPHHVDFVZ5KDEQAQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176392132 (31m5.215965252s) [noise=true , address=45.91.116.13:4155 , piece_id=EBZOXPRXUFYQ5A62EMLSKT5VHJ2HGHRCNXMETP6LED4COFPUZVXA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176733806 (31m5.216306926s) [noise=true , address=45.91.116.11:4119 , piece_id=6ZF5PBUSVQ622T27KYYVWAHJCJUCCLHHGSK3YQPMOMPL6WEJFLFQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176903921 (31m5.216477041s) [noise=true , address=45.91.116.4:4938 , piece_id=MA4DUBDNGDBN2QDPJNRSHZNOMLELAVYJOJTLSSA2GTEWDZU2HIVA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177129533 (31m5.216702653s) [noise=true , address=45.91.116.8:4045 , piece_id=H2QQ2QDAP6GTKITG7CRR7SDCURHZ4OSIIYFNFAX6KMALRMT2WN5A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177450557 (31m5.217023667s) [noise=true , address=45.91.116.6:4973 , piece_id=JQWN3PO5VQ4CWK3BOFEXMPXWGJWFSM4TCBJWR6DAMNLQYOAYP3AQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177754920 (31m5.21732804s) [noise=true , address=45.91.116.14:4189 , piece_id=2LI5UK7J6S66I2ISJEFLLZ4JWLRXNH4QZAHICOH25Z7YAVBQZ2JA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177779867 (31m5.217352987s) [noise=true , address=45.91.116.10:4081 , piece_id=2CVTVPEZIYIP4QUPRKNQ27MYWVTZUESLE466EDUGPLFM6V32W7DA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178107094 (31m5.217680224s) [noise=true , address=45.91.116.12:4227 , piece_id=45X4BYU4ARSF65AYRRUDOGIE3YCHZL6XV2UG76WMVV6KLSMRYOUQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178122282 (31m5.217695402s) [noise=true , address=77.90.31.13:4627 , piece_id=QSHHL4FUS4B3HITSAQRLDFWRQ35QMRKJOM5NPLQ4KAPYUODK7GRA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178156478 (31m5.217729598s) [noise=true , address=45.91.116.8:4048 , piece_id=COHGPSHEZMEQRTS5KM3UXGLEOGDPBJIIPJTNA54II7L3HRFJBRMQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178409793 (31m5.217982913s) [noise=true , address=45.91.116.6:4975 , piece_id=6E3HA6RCAZAACOOQ6RFZB73C4VAFPVNN4Z2P7CGE4ND2IJEKONUA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.183699795 (31m5.223272905s) [noise=true , address=77.90.31.15:4697 , piece_id=5Q7YYNYI5NYN5HGKJ74RHAYZMAAAWJYEHYXX2VKD7NWF3FNB7J4A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.183980152 (31m5.223553272s) [noise=true , address=45.91.116.7:4009 , piece_id=3ZJHMECXUCBY4ZG7KYFNK4AFT4SKZ2SCFI3XCCUHWGNIRSFA6A6A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.184265027 (31m5.223838147s) [noise=true , address=45.91.116.13:4154 , piece_id=2BYNORNSHZBRAOUU2YBWCBOWGJHXAJT6CXVUPNLZRJGIH2TCJ26A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.184401618 (31m5.223974738s) [noise=true , address=45.91.116.8:4047 , piece_id=YRFF7ABNSHO4G2VUT7LYAZZ3XX7LDORBNL76TUZBWXOYUR43RNYA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:30.987894621 (31m7.027467741s) [upload=445]: Next returned nil segment Sep 6 13:59:30.987918496 (31m7.027491667s) [upload=445]: Waiting for error group managing segments... Sep 6 13:59:31.295099475 (31m7.334672595s) [noise=true , address=45.91.116.8:4045 , piece_id=H2QQ2QDAP6GTKITG7CRR7SDCURHZ4OSIIYFNFAX6KMALRMT2WN5A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.334237326 (31m7.373810446s) [noise=true , address=45.91.116.4:4938 , piece_id=MA4DUBDNGDBN2QDPJNRSHZNOMLELAVYJOJTLSSA2GTEWDZU2HIVA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.351472346 (31m7.391045466s) [noise=true , address=45.91.116.6:4973 , piece_id=JQWN3PO5VQ4CWK3BOFEXMPXWGJWFSM4TCBJWR6DAMNLQYOAYP3AQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.357682198 (31m7.397255318s) [noise=true , address=45.91.116.11:4119 , piece_id=6ZF5PBUSVQ622T27KYYVWAHJCJUCCLHHGSK3YQPMOMPL6WEJFLFQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.376896446 (31m7.416469566s) [noise=true , address=45.91.116.14:4189 , piece_id=2LI5UK7J6S66I2ISJEFLLZ4JWLRXNH4QZAHICOH25Z7YAVBQZ2JA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.379670153 (31m7.419243273s) [noise=true , address=45.91.116.7:4009 , piece_id=3ZJHMECXUCBY4ZG7KYFNK4AFT4SKZ2SCFI3XCCUHWGNIRSFA6A6A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.402121034 (31m7.441694164s) [noise=true , address=45.91.116.13:4154 , piece_id=2BYNORNSHZBRAOUU2YBWCBOWGJHXAJT6CXVUPNLZRJGIH2TCJ26A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.413862341 (31m7.453435451s) [noise=true , address=45.91.116.6:4975 , piece_id=6E3HA6RCAZAACOOQ6RFZB73C4VAFPVNN4Z2P7CGE4ND2IJEKONUA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.415618231 (31m7.455191351s) [noise=true , address=45.91.116.8:4047 , piece_id=YRFF7ABNSHO4G2VUT7LYAZZ3XX7LDORBNL76TUZBWXOYUR43RNYA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.453979285 (31m7.493552405s) [noise=true , address=45.91.116.12:4227 , piece_id=45X4BYU4ARSF65AYRRUDOGIE3YCHZL6XV2UG76WMVV6KLSMRYOUQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.458707914 (31m7.498281034s) [noise=true , address=45.91.116.10:4081 , piece_id=2CVTVPEZIYIP4QUPRKNQ27MYWVTZUESLE466EDUGPLFM6V32W7DA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.495016151 (31m7.534589271s) [noise=true , address=77.90.31.13:4627 , piece_id=QSHHL4FUS4B3HITSAQRLDFWRQ35QMRKJOM5NPLQ4KAPYUODK7GRA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.547548343 (31m7.587121463s) [noise=true , address=45.91.116.10:4082 , piece_id=XBRBJBJFXVUJQETVTXTV6ZTULWTM2WWAJIQDDPHHVDFVZ5KDEQAQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.547579943 (31m7.587153063s) [seg_pos={0 0}, upload=445]: Segment reached optimal threshold of 13 pieces. Sep 6 13:59:31.547726283 (31m7.587299403s) [noise=true , address=77.90.31.15:4697 , piece_id=5Q7YYNYI5NYN5HGKJ74RHAYZMAAAWJYEHYXX2VKD7NWF3FNB7J4A, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.547891519 (31m7.587464639s) [noise=true , address=45.91.116.13:4155 , piece_id=EBZOXPRXUFYQ5A62EMLSKT5VHJ2HGHRCNXMETP6LED4COFPUZVXA, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.547945783 (31m7.587518903s) [noise=true , address=45.91.116.8:4048 , piece_id=COHGPSHEZMEQRTS5KM3UXGLEOGDPBJIIPJTNA54II7L3HRFJBRMQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.548004926 (31m7.587578046s) [seg_pos={0 0}, upload=445]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:59:31.548083657 (31m7.587656777s) [upload=445]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:59:31.548089287 (31m7.587662407s) [upload=445]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:59:31.891122130 (31m7.93069525s) [upload=445]: Done uploading segments... Sep 6 13:59:31.891143722 (31m7.930716842s) [upload=445]: Upload finished. err: <nil> Duration: 18.184076779s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 16 out of 60 is failed (threshold=3) [....................XX......XX........XXX...XX...XX..XXX..XX] duration: 30.002673931s FAIL (2025-09-06 13:59:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:59:28.826951007 (31m4.866524127s) [upload=445]: Starting upload Sep 6 13:59:28.827047741 (31m4.866620881s) [upload=445]: Done starting upload Sep 6 13:59:28.827275718 (31m4.866848848s) [upload=445]: Uploading segments... Sep 6 13:59:28.827389937 (31m4.866963057s) [upload=445]: Got next segment. Inline: false Sep 6 13:59:28.827427438 (31m4.867000568s) [upload=445]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:59:28.827446144 (31m4.867019264s) [upload=445]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:59:29.176243838 (31m5.215816958s) [seg_pos={0 0}, upload=445]: Begin upload segment... Sep 6 13:59:29.176289976 (31m5.215863096s) [seg_pos={0 0}, upload=445]: Done begin upload segment. Sep 6 13:59:29.176311858 (31m5.215884978s) [noise=true , address=45.91.116.10:4082 , piece_id=XBRBJBJFXVUJQETVTXTV6ZTULWTM2WWAJIQDDPHHVDFVZ5KDEQAQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176392132 (31m5.215965252s) [noise=true , address=45.91.116.13:4155 , piece_id=EBZOXPRXUFYQ5A62EMLSKT5VHJ2HGHRCNXMETP6LED4COFPUZVXA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176733806 (31m5.216306926s) [noise=true , address=45.91.116.11:4119 , piece_id=6ZF5PBUSVQ622T27KYYVWAHJCJUCCLHHGSK3YQPMOMPL6WEJFLFQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.176903921 (31m5.216477041s) [noise=true , address=45.91.116.4:4938 , piece_id=MA4DUBDNGDBN2QDPJNRSHZNOMLELAVYJOJTLSSA2GTEWDZU2HIVA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177129533 (31m5.216702653s) [noise=true , address=45.91.116.8:4045 , piece_id=H2QQ2QDAP6GTKITG7CRR7SDCURHZ4OSIIYFNFAX6KMALRMT2WN5A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177450557 (31m5.217023667s) [noise=true , address=45.91.116.6:4973 , piece_id=JQWN3PO5VQ4CWK3BOFEXMPXWGJWFSM4TCBJWR6DAMNLQYOAYP3AQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177754920 (31m5.21732804s) [noise=true , address=45.91.116.14:4189 , piece_id=2LI5UK7J6S66I2ISJEFLLZ4JWLRXNH4QZAHICOH25Z7YAVBQZ2JA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.177779867 (31m5.217352987s) [noise=true , address=45.91.116.10:4081 , piece_id=2CVTVPEZIYIP4QUPRKNQ27MYWVTZUESLE466EDUGPLFM6V32W7DA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178107094 (31m5.217680224s) [noise=true , address=45.91.116.12:4227 , piece_id=45X4BYU4ARSF65AYRRUDOGIE3YCHZL6XV2UG76WMVV6KLSMRYOUQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178122282 (31m5.217695402s) [noise=true , address=77.90.31.13:4627 , piece_id=QSHHL4FUS4B3HITSAQRLDFWRQ35QMRKJOM5NPLQ4KAPYUODK7GRA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178156478 (31m5.217729598s) [noise=true , address=45.91.116.8:4048 , piece_id=COHGPSHEZMEQRTS5KM3UXGLEOGDPBJIIPJTNA54II7L3HRFJBRMQ, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.178409793 (31m5.217982913s) [noise=true , address=45.91.116.6:4975 , piece_id=6E3HA6RCAZAACOOQ6RFZB73C4VAFPVNN4Z2P7CGE4ND2IJEKONUA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.183699795 (31m5.223272905s) [noise=true , address=77.90.31.15:4697 , piece_id=5Q7YYNYI5NYN5HGKJ74RHAYZMAAAWJYEHYXX2VKD7NWF3FNB7J4A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.183980152 (31m5.223553272s) [noise=true , address=45.91.116.7:4009 , piece_id=3ZJHMECXUCBY4ZG7KYFNK4AFT4SKZ2SCFI3XCCUHWGNIRSFA6A6A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.184265027 (31m5.223838147s) [noise=true , address=45.91.116.13:4154 , piece_id=2BYNORNSHZBRAOUU2YBWCBOWGJHXAJT6CXVUPNLZRJGIH2TCJ26A, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:29.184401618 (31m5.223974738s) [noise=true , address=45.91.116.8:4047 , piece_id=YRFF7ABNSHO4G2VUT7LYAZZ3XX7LDORBNL76TUZBWXOYUR43RNYA, seg_pos={0 0}, upload=445]: Uploading piece... Sep 6 13:59:30.987894621 (31m7.027467741s) [upload=445]: Next returned nil segment Sep 6 13:59:30.987918496 (31m7.027491667s) [upload=445]: Waiting for error group managing segments... Sep 6 13:59:31.295099475 (31m7.334672595s) [noise=true , address=45.91.116.8:4045 , piece_id=H2QQ2QDAP6GTKITG7CRR7SDCURHZ4OSIIYFNFAX6KMALRMT2WN5A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.334237326 (31m7.373810446s) [noise=true , address=45.91.116.4:4938 , piece_id=MA4DUBDNGDBN2QDPJNRSHZNOMLELAVYJOJTLSSA2GTEWDZU2HIVA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.351472346 (31m7.391045466s) [noise=true , address=45.91.116.6:4973 , piece_id=JQWN3PO5VQ4CWK3BOFEXMPXWGJWFSM4TCBJWR6DAMNLQYOAYP3AQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.357682198 (31m7.397255318s) [noise=true , address=45.91.116.11:4119 , piece_id=6ZF5PBUSVQ622T27KYYVWAHJCJUCCLHHGSK3YQPMOMPL6WEJFLFQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.376896446 (31m7.416469566s) [noise=true , address=45.91.116.14:4189 , piece_id=2LI5UK7J6S66I2ISJEFLLZ4JWLRXNH4QZAHICOH25Z7YAVBQZ2JA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.379670153 (31m7.419243273s) [noise=true , address=45.91.116.7:4009 , piece_id=3ZJHMECXUCBY4ZG7KYFNK4AFT4SKZ2SCFI3XCCUHWGNIRSFA6A6A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.402121034 (31m7.441694164s) [noise=true , address=45.91.116.13:4154 , piece_id=2BYNORNSHZBRAOUU2YBWCBOWGJHXAJT6CXVUPNLZRJGIH2TCJ26A, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.413862341 (31m7.453435451s) [noise=true , address=45.91.116.6:4975 , piece_id=6E3HA6RCAZAACOOQ6RFZB73C4VAFPVNN4Z2P7CGE4ND2IJEKONUA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.415618231 (31m7.455191351s) [noise=true , address=45.91.116.8:4047 , piece_id=YRFF7ABNSHO4G2VUT7LYAZZ3XX7LDORBNL76TUZBWXOYUR43RNYA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.453979285 (31m7.493552405s) [noise=true , address=45.91.116.12:4227 , piece_id=45X4BYU4ARSF65AYRRUDOGIE3YCHZL6XV2UG76WMVV6KLSMRYOUQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.458707914 (31m7.498281034s) [noise=true , address=45.91.116.10:4081 , piece_id=2CVTVPEZIYIP4QUPRKNQ27MYWVTZUESLE466EDUGPLFM6V32W7DA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.495016151 (31m7.534589271s) [noise=true , address=77.90.31.13:4627 , piece_id=QSHHL4FUS4B3HITSAQRLDFWRQ35QMRKJOM5NPLQ4KAPYUODK7GRA, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.547548343 (31m7.587121463s) [noise=true , address=45.91.116.10:4082 , piece_id=XBRBJBJFXVUJQETVTXTV6ZTULWTM2WWAJIQDDPHHVDFVZ5KDEQAQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: <nil> Sep 6 13:59:31.547579943 (31m7.587153063s) [seg_pos={0 0}, upload=445]: Segment reached optimal threshold of 13 pieces. Sep 6 13:59:31.547726283 (31m7.587299403s) [noise=true , address=77.90.31.15:4697 , piece_id=5Q7YYNYI5NYN5HGKJ74RHAYZMAAAWJYEHYXX2VKD7NWF3FNB7J4A, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AwQ11hGrJg7dmGK4WkceFmt6VSXLC7FXgXH159pCAe2muwps5): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.547891519 (31m7.587464639s) [noise=true , address=45.91.116.13:4155 , piece_id=EBZOXPRXUFYQ5A62EMLSKT5VHJ2HGHRCNXMETP6LED4COFPUZVXA, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.547945783 (31m7.587518903s) [noise=true , address=45.91.116.8:4048 , piece_id=COHGPSHEZMEQRTS5KM3UXGLEOGDPBJIIPJTNA54II7L3HRFJBRMQ, seg_pos={0 0}, upload=445]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 6 13:59:31.548004926 (31m7.587578046s) [seg_pos={0 0}, upload=445]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:59:31.548083657 (31m7.587656777s) [upload=445]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:59:31.548089287 (31m7.587662407s) [upload=445]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:59:31.891122130 (31m7.93069525s) [upload=445]: Done uploading segments... Sep 6 13:59:31.891143722 (31m7.930716842s) [upload=445]: Upload finished. err: <nil> Duration: 30.0026949s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 15 out of 60 is failed (threshold=3) [.....................XX......XX........XXX...XX...XX..XXX..X] duration: 30.002817817s FAIL (2025-09-06 13:59:12 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:58:42.764653947 (30m18.804227067s) [upload=430]: Starting upload Sep 6 13:58:42.764705064 (30m18.804278184s) [upload=430]: Done starting upload Sep 6 13:58:42.764884929 (30m18.804458049s) [upload=430]: Uploading segments... Sep 6 13:58:42.764922931 (30m18.804496041s) [upload=430]: Got next segment. Inline: false Sep 6 13:58:42.764953741 (30m18.804526861s) [upload=430]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:58:42.764959302 (30m18.804532422s) [upload=430]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:58:43.287033204 (30m19.326606324s) [seg_pos={0 0}, upload=430]: Begin upload segment... Sep 6 13:58:43.287067790 (30m19.3266409s) [seg_pos={0 0}, upload=430]: Done begin upload segment. Sep 6 13:58:43.287086536 (30m19.326659656s) [noise=true , address=77.90.31.21:4878 , piece_id=WFBSM5ELVHRHVHTWND7PP2GROOCPFFQ5B2YCRFUHEQLD3QMZFLCQ, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.287507672 (30m19.327080782s) [noise=true , address=45.91.116.5:4297 , piece_id=5WVKDZNXMK673BTZZCKVZKC6NT6Y6HMDOUKWZHHVVC4OB7IYBN7Q, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.287793578 (30m19.327366698s) [noise=true , address=45.91.116.6:4973 , piece_id=U3PJZWWFRBTHK3HJBBT4CAO4QDUXPNS3VUZMTIGJ7MOY57EIQLFQ, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.288060269 (30m19.327633389s) [noise=true , address=77.90.31.18:4806 , piece_id=TZL2THGO3EGNCLOFPV772VHVKWBSDR6MN455ZPIRWZTGIYLAFE4A, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.288329695 (30m19.327902815s) [noise=true , address=45.91.116.12:4226 , piece_id=6VFE2S4V7RUSWLI336OTZOQ6MP4ADXMSYC4T64DMKUAAWF2DUTVA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.288566989 (30m19.328140109s) [noise=true , address=45.91.116.8:4047 , piece_id=K2BD43DXIIUGRB46GMB6LFJRZRHOG67IO2AAQUCCFFUYGFVQ44TQ, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.288781099 (30m19.328354219s) [noise=true , address=45.91.116.14:4189 , piece_id=NT4EJKUL4SYNRELTJ625Y6N7X56CUVWI3KHJM6VYHSEMS6PJJERA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.289657486 (30m19.329230606s) [noise=true , address=77.90.31.13:4626 , piece_id=4DKLNL5T5NE55S22FX2K2MLC4ROOFFHBKDE67H7YZZJOVGA3G2HA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.289895481 (30m19.329468611s) [noise=true , address=77.90.31.16:4734 , piece_id=KXXCJQFJ7RS6JWRY5H6463HJVOTIAQ75425EMU7UJVRMBQYRU2QQ, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.290146330 (30m19.32971945s) [noise=true , address=45.91.116.4:4939 , piece_id=VLQ6SWCIEVQLWOVSHKHHHWM6AZSBCMLHEQZF5FVVOHDPAQRTCCAA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.290476954 (30m19.330050074s) [noise=true , address=45.91.116.10:4082 , piece_id=K6TGJSBLMO5I5G2LYAJYUVDDI2C4O5SRNPWUAFQZZC3IFQRJNFLA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.290756428 (30m19.330329548s) [noise=true , address=77.90.31.13:4628 , piece_id=7PEJPCTNKA5JKDYEBH5QASO7YMZDIKB6KFU7NBMPCCZVZD4UDBPA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.290987430 (30m19.33056055s) [noise=true , address=45.91.116.11:4119 , piece_id=4OCP2PO2MTNRTFFZMHI2BNIW4XVADMLYBPILE3VHLNPK3KUTZ3JA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.291186271 (30m19.330759391s) [noise=true , address=77.90.31.19:4841 , piece_id=P7QB3VDGXIUDPCS24PNBNT5BJDNL4WZCNA7VWWB5NDXWGEM3CL4A, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.291445397 (30m19.331018517s) [noise=true , address=77.90.31.19:4843 , piece_id=VTPPT6UHDRJPSRLMT62G2PGEGQFOMZ6I3T4Y2JHSYVCG6UUBWWEA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:43.291642864 (30m19.331215984s) [noise=true , address=45.91.116.10:4083 , piece_id=OGRYI5IC2CEGRKSAWQHZVKOFL7LEKPCJGLG5S65W6SU3ZHTD66XA, seg_pos={0 0}, upload=430]: Uploading piece... Sep 6 13:58:45.152636636 (30m21.192209756s) [upload=430]: Next returned nil segment Sep 6 13:58:45.152655352 (30m21.192228472s) [upload=430]: Waiting for error group managing segments... Sep 6 13:58:45.466940775 (30m21.506513895s) [noise=true , address=77.90.31.13:4626 , piece_id=4DKLNL5T5NE55S22FX2K2MLC4ROOFFHBKDE67H7YZZJOVGA3G2HA, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.472862567 (30m21.512435687s) [noise=true , address=77.90.31.13:4628 , piece_id=7PEJPCTNKA5JKDYEBH5QASO7YMZDIKB6KFU7NBMPCCZVZD4UDBPA, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.477672641 (30m21.517245761s) [noise=true , address=45.91.116.12:4226 , piece_id=6VFE2S4V7RUSWLI336OTZOQ6MP4ADXMSYC4T64DMKUAAWF2DUTVA, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.478296986 (30m21.517870106s) [noise=true , address=45.91.116.10:4082 , piece_id=K6TGJSBLMO5I5G2LYAJYUVDDI2C4O5SRNPWUAFQZZC3IFQRJNFLA, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.479145269 (30m21.518718389s) [noise=true , address=45.91.116.6:4973 , piece_id=U3PJZWWFRBTHK3HJBBT4CAO4QDUXPNS3VUZMTIGJ7MOY57EIQLFQ, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.506571510 (30m21.54614463s) [noise=true , address=77.90.31.16:4734 , piece_id=KXXCJQFJ7RS6JWRY5H6463HJVOTIAQ75425EMU7UJVRMBQYRU2QQ, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.511996861 (30m21.551569981s) [noise=true , address=45.91.116.14:4189 , piece_id=NT4EJKUL4SYNRELTJ625Y6N7X56CUVWI3KHJM6VYHSEMS6PJJERA, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.518233856 (30m21.557806976s) [noise=true , address=77.90.31.21:4878 , piece_id=WFBSM5ELVHRHVHTWND7PP2GROOCPFFQ5B2YCRFUHEQLD3QMZFLCQ, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.578630978 (30m21.618204108s) [noise=true , address=45.91.116.5:4297 , piece_id=5WVKDZNXMK673BTZZCKVZKC6NT6Y6HMDOUKWZHHVVC4OB7IYBN7Q, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.580029574 (30m21.619602694s) [noise=true , address=77.90.31.19:4843 , piece_id=VTPPT6UHDRJPSRLMT62G2PGEGQFOMZ6I3T4Y2JHSYVCG6UUBWWEA, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.591562823 (30m21.631135943s) [noise=true , address=77.90.31.19:4841 , piece_id=P7QB3VDGXIUDPCS24PNBNT5BJDNL4WZCNA7VWWB5NDXWGEM3CL4A, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.593168376 (30m21.632741496s) [noise=true , address=77.90.31.18:4806 , piece_id=TZL2THGO3EGNCLOFPV772VHVKWBSDR6MN455ZPIRWZTGIYLAFE4A, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.607743585 (30m21.647316705s) [noise=true , address=45.91.116.8:4047 , piece_id=K2BD43DXIIUGRB46GMB6LFJRZRHOG67IO2AAQUCCFFUYGFVQ44TQ, seg_pos={0 0}, upload=430]: Done uploading piece. err: <nil> Sep 6 13:58:45.607759977 (30m21.647333087s) [seg_pos={0 0}, upload=430]: Segment reached optimal threshold of 13 pieces. Sep 6 13:58:45.608035043 (30m21.647608163s) [noise=true , address=45.91.116.4:4939 , piece_id=VLQ6SWCIEVQLWOVSHKHHHWM6AZSBCMLHEQZF5FVVOHDPAQRTCCAA, seg_pos={0 0}, upload=430]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX): protocol: write tcp 188.166.225.170:34020->45.91.116.4:4939: use of closed network connection; write tcp 188.166.225.170:34020->45.91.116.4:4939: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34020->45.91.116.4:4939: use of closed network connection Sep 6 13:58:45.608082263 (30m21.647655383s) [noise=true , address=45.91.116.10:4083 , piece_id=OGRYI5IC2CEGRKSAWQHZVKOFL7LEKPCJGLG5S65W6SU3ZHTD66XA, seg_pos={0 0}, upload=430]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: write tcp 188.166.225.170:46668->45.91.116.10:4083: use of closed network connection; write tcp 188.166.225.170:46668->45.91.116.10:4083: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46668->45.91.116.10:4083: use of closed network connection Sep 6 13:58:45.608116690 (30m21.64768981s) [noise=true , address=45.91.116.11:4119 , piece_id=4OCP2PO2MTNRTFFZMHI2BNIW4XVADMLYBPILE3VHLNPK3KUTZ3JA, seg_pos={0 0}, upload=430]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:45418->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:45418->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45418->45.91.116.11:4119: use of closed network connection Sep 6 13:58:45.608125716 (30m21.647698837s) [seg_pos={0 0}, upload=430]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:58:45.608180611 (30m21.647753732s) [upload=430]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:58:45.608184148 (30m21.647757268s) [upload=430]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:58:45.962808111 (30m22.002381231s) [upload=430]: Done uploading segments... Sep 6 13:58:45.962839421 (30m22.002412541s) [upload=430]: Upload finished. err: <nil> Duration: 30.00284101s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [......................XX......XX........XXX...XX...XX..XXX..] duration: 15.87479128s FAIL (2025-09-06 13:58:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:56:11.784228618 (27m47.823801738s) [upload=399]: Starting upload Sep 6 13:56:11.784284775 (27m47.823857895s) [upload=399]: Done starting upload Sep 6 13:56:11.784435193 (27m47.824008313s) [upload=399]: Uploading segments... Sep 6 13:56:11.784479398 (27m47.824052508s) [upload=399]: Got next segment. Inline: false Sep 6 13:56:11.784498935 (27m47.824072055s) [upload=399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:56:11.784503313 (27m47.824076433s) [upload=399]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:56:12.025562526 (27m48.065135646s) [seg_pos={0 0}, upload=399]: Begin upload segment... Sep 6 13:56:12.025614836 (27m48.065187956s) [seg_pos={0 0}, upload=399]: Done begin upload segment. Sep 6 13:56:12.025649193 (27m48.065222313s) [noise=true , address=77.90.31.10:4518 , piece_id=PXX3UUN25OUY3PMLSOY2RYI5V3S2XC7D3YDFLKLLAT77NIR6B5CQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026146965 (27m48.065720075s) [noise=true , address=77.90.31.15:4698 , piece_id=FKDMDFVF6T7IDOCWEND7JIQYHEA7QAZ7QT3BVMB6HB4INDP4BACA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026488648 (27m48.066061768s) [noise=true , address=45.91.116.8:4047 , piece_id=GLNQE64DXUGQQDOPBKUEVTQUPYKIY36OXYY3AT2VU4CVD32WNWOA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026825463 (27m48.066398583s) [noise=true , address=45.91.116.11:4118 , piece_id=S5M4G5WK6PKHVHBYZGIW7YNZEQUPRLY4VPWANNAT7247K5DFQWVA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.027414019 (27m48.066987139s) [noise=true , address=77.90.31.12:4590 , piece_id=QA7A6MUVOUPSUK74PGWAIDSK6PKCKSDWFWDCTENJ4273WOWWWRWA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028280808 (27m48.067853928s) [noise=true , address=45.91.116.5:4298 , piece_id=GG6YIMNE6SRNYEWZSY2U4KY27YEYUH2XUXNFHXRGOKWIJD2BDL4Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028527510 (27m48.06810063s) [noise=true , address=45.91.116.14:4189 , piece_id=BNDUXGDW2EPEPJXZANLOOZG2LUIOVU3EI5MCG6QZLLEA52HFTD4A, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028762650 (27m48.06833577s) [noise=true , address=77.90.31.19:4841 , piece_id=QGSCJFC7LLNVWVEH2G7OBYMCSCPDMFNB74OHB75725Z3T7R76OUA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029007118 (27m48.068580238s) [noise=true , address=45.91.116.4:4938 , piece_id=DJVQ5EZUJVPJNQYXKLTVPJ2WEFYTUI42AALAODWQJSGVPYGXCUXQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029314787 (27m48.068887917s) [noise=true , address=77.90.31.17:4772 , piece_id=ES2I34T7P4MBDFYVDIPCT4WHEZ7CLMYGEWSEA642KJ3DPS7Z2Q5Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029568472 (27m48.069141592s) [noise=true , address=77.90.31.16:4734 , piece_id=5E4RJK2DXZKN554GPNRWMDBQHGNYNV6XYUOF6PKTFVVX4TSG5XHA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029846073 (27m48.069419183s) [noise=true , address=77.90.31.15:4699 , piece_id=HO3DW4BYBCXHNLZBIDJQQ6AE7AYTDE6NXVE3YVZU3MJWFOFVWFZA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030161848 (27m48.069734968s) [noise=true , address=45.91.116.8:4045 , piece_id=FT3O2Q34T34CUEW2RYTCBQYLXMFFXMJJG4K3CTHCSTVVICGNOAOQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030396566 (27m48.069969686s) [noise=true , address=45.91.116.6:4973 , piece_id=CPLW4X7AI7FLX3SVEBWCWL35THGBVFU37NRX4P2LC36Z3ZXYVULA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030629903 (27m48.070203023s) [noise=true , address=45.91.116.10:4083 , piece_id=X3WQ622EA6UWZUWRGZ3745BHZZPIXRJFFNZIW4NMAF6G56ZIRVZA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.031224090 (27m48.07079721s) [noise=true , address=77.90.31.19:4843 , piece_id=A2PW54R4F2HRUCX7TQSEUD7JPX65FQE5MLA5A2RK2Q6MVNMYOL3Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:13.883361725 (27m49.922934845s) [upload=399]: Next returned nil segment Sep 6 13:56:13.883376664 (27m49.922949784s) [upload=399]: Waiting for error group managing segments... Sep 6 13:56:14.173140084 (27m50.212713204s) [noise=true , address=45.91.116.10:4083 , piece_id=X3WQ622EA6UWZUWRGZ3745BHZZPIXRJFFNZIW4NMAF6G56ZIRVZA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.220496311 (27m50.260069431s) [noise=true , address=45.91.116.5:4298 , piece_id=GG6YIMNE6SRNYEWZSY2U4KY27YEYUH2XUXNFHXRGOKWIJD2BDL4Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.255626334 (27m50.295199454s) [noise=true , address=77.90.31.19:4843 , piece_id=A2PW54R4F2HRUCX7TQSEUD7JPX65FQE5MLA5A2RK2Q6MVNMYOL3Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.263389889 (27m50.302963009s) [noise=true , address=77.90.31.12:4590 , piece_id=QA7A6MUVOUPSUK74PGWAIDSK6PKCKSDWFWDCTENJ4273WOWWWRWA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.287035495 (27m50.326608615s) [noise=true , address=77.90.31.19:4841 , piece_id=QGSCJFC7LLNVWVEH2G7OBYMCSCPDMFNB74OHB75725Z3T7R76OUA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.292975913 (27m50.332549043s) [noise=true , address=45.91.116.8:4045 , piece_id=FT3O2Q34T34CUEW2RYTCBQYLXMFFXMJJG4K3CTHCSTVVICGNOAOQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.303591796 (27m50.343164916s) [noise=true , address=45.91.116.4:4938 , piece_id=DJVQ5EZUJVPJNQYXKLTVPJ2WEFYTUI42AALAODWQJSGVPYGXCUXQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.342732192 (27m50.382305312s) [noise=true , address=45.91.116.8:4047 , piece_id=GLNQE64DXUGQQDOPBKUEVTQUPYKIY36OXYY3AT2VU4CVD32WNWOA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.384729616 (27m50.424302726s) [noise=true , address=77.90.31.16:4734 , piece_id=5E4RJK2DXZKN554GPNRWMDBQHGNYNV6XYUOF6PKTFVVX4TSG5XHA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.411143620 (27m50.45071674s) [noise=true , address=77.90.31.15:4698 , piece_id=FKDMDFVF6T7IDOCWEND7JIQYHEA7QAZ7QT3BVMB6HB4INDP4BACA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.447251863 (27m50.486824984s) [noise=true , address=45.91.116.6:4973 , piece_id=CPLW4X7AI7FLX3SVEBWCWL35THGBVFU37NRX4P2LC36Z3ZXYVULA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.456897671 (27m50.496470791s) [noise=true , address=77.90.31.10:4518 , piece_id=PXX3UUN25OUY3PMLSOY2RYI5V3S2XC7D3YDFLKLLAT77NIR6B5CQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.490628396 (27m50.530201516s) [noise=true , address=77.90.31.17:4772 , piece_id=ES2I34T7P4MBDFYVDIPCT4WHEZ7CLMYGEWSEA642KJ3DPS7Z2Q5Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.490655378 (27m50.530228498s) [seg_pos={0 0}, upload=399]: Segment reached optimal threshold of 13 pieces. Sep 6 13:56:14.490779695 (27m50.530352805s) [noise=true , address=45.91.116.14:4189 , piece_id=BNDUXGDW2EPEPJXZANLOOZG2LUIOVU3EI5MCG6QZLLEA52HFTD4A, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF Sep 6 13:56:14.490926587 (27m50.530499707s) [noise=true , address=45.91.116.11:4118 , piece_id=S5M4G5WK6PKHVHBYZGIW7YNZEQUPRLY4VPWANNAT7247K5DFQWVA, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 13:56:14.491022340 (27m50.53059545s) [noise=true , address=77.90.31.15:4699 , piece_id=HO3DW4BYBCXHNLZBIDJQQ6AE7AYTDE6NXVE3YVZU3MJWFOFVWFZA, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: read tcp 188.166.225.170:45032->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 13:56:14.491037539 (27m50.530610659s) [seg_pos={0 0}, upload=399]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:56:14.491121650 (27m50.53069477s) [upload=399]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:56:14.491126880 (27m50.5307s ) [upload=399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:56:14.796736622 (27m50.836309742s) [upload=399]: Done uploading segments... Sep 6 13:56:14.796756030 (27m50.83632916s) [upload=399]: Upload finished. err: <nil> Duration: 15.87482355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [.......................XX......XX........XXX...XX...XX..XXX.] duration: 10.221384162s FAIL (2025-09-06 13:57:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:56:11.784228618 (27m47.823801738s) [upload=399]: Starting upload Sep 6 13:56:11.784284775 (27m47.823857895s) [upload=399]: Done starting upload Sep 6 13:56:11.784435193 (27m47.824008313s) [upload=399]: Uploading segments... Sep 6 13:56:11.784479398 (27m47.824052508s) [upload=399]: Got next segment. Inline: false Sep 6 13:56:11.784498935 (27m47.824072055s) [upload=399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:56:11.784503313 (27m47.824076433s) [upload=399]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:56:12.025562526 (27m48.065135646s) [seg_pos={0 0}, upload=399]: Begin upload segment... Sep 6 13:56:12.025614836 (27m48.065187956s) [seg_pos={0 0}, upload=399]: Done begin upload segment. Sep 6 13:56:12.025649193 (27m48.065222313s) [noise=true , address=77.90.31.10:4518 , piece_id=PXX3UUN25OUY3PMLSOY2RYI5V3S2XC7D3YDFLKLLAT77NIR6B5CQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026146965 (27m48.065720075s) [noise=true , address=77.90.31.15:4698 , piece_id=FKDMDFVF6T7IDOCWEND7JIQYHEA7QAZ7QT3BVMB6HB4INDP4BACA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026488648 (27m48.066061768s) [noise=true , address=45.91.116.8:4047 , piece_id=GLNQE64DXUGQQDOPBKUEVTQUPYKIY36OXYY3AT2VU4CVD32WNWOA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026825463 (27m48.066398583s) [noise=true , address=45.91.116.11:4118 , piece_id=S5M4G5WK6PKHVHBYZGIW7YNZEQUPRLY4VPWANNAT7247K5DFQWVA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.027414019 (27m48.066987139s) [noise=true , address=77.90.31.12:4590 , piece_id=QA7A6MUVOUPSUK74PGWAIDSK6PKCKSDWFWDCTENJ4273WOWWWRWA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028280808 (27m48.067853928s) [noise=true , address=45.91.116.5:4298 , piece_id=GG6YIMNE6SRNYEWZSY2U4KY27YEYUH2XUXNFHXRGOKWIJD2BDL4Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028527510 (27m48.06810063s) [noise=true , address=45.91.116.14:4189 , piece_id=BNDUXGDW2EPEPJXZANLOOZG2LUIOVU3EI5MCG6QZLLEA52HFTD4A, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028762650 (27m48.06833577s) [noise=true , address=77.90.31.19:4841 , piece_id=QGSCJFC7LLNVWVEH2G7OBYMCSCPDMFNB74OHB75725Z3T7R76OUA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029007118 (27m48.068580238s) [noise=true , address=45.91.116.4:4938 , piece_id=DJVQ5EZUJVPJNQYXKLTVPJ2WEFYTUI42AALAODWQJSGVPYGXCUXQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029314787 (27m48.068887917s) [noise=true , address=77.90.31.17:4772 , piece_id=ES2I34T7P4MBDFYVDIPCT4WHEZ7CLMYGEWSEA642KJ3DPS7Z2Q5Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029568472 (27m48.069141592s) [noise=true , address=77.90.31.16:4734 , piece_id=5E4RJK2DXZKN554GPNRWMDBQHGNYNV6XYUOF6PKTFVVX4TSG5XHA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029846073 (27m48.069419183s) [noise=true , address=77.90.31.15:4699 , piece_id=HO3DW4BYBCXHNLZBIDJQQ6AE7AYTDE6NXVE3YVZU3MJWFOFVWFZA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030161848 (27m48.069734968s) [noise=true , address=45.91.116.8:4045 , piece_id=FT3O2Q34T34CUEW2RYTCBQYLXMFFXMJJG4K3CTHCSTVVICGNOAOQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030396566 (27m48.069969686s) [noise=true , address=45.91.116.6:4973 , piece_id=CPLW4X7AI7FLX3SVEBWCWL35THGBVFU37NRX4P2LC36Z3ZXYVULA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030629903 (27m48.070203023s) [noise=true , address=45.91.116.10:4083 , piece_id=X3WQ622EA6UWZUWRGZ3745BHZZPIXRJFFNZIW4NMAF6G56ZIRVZA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.031224090 (27m48.07079721s) [noise=true , address=77.90.31.19:4843 , piece_id=A2PW54R4F2HRUCX7TQSEUD7JPX65FQE5MLA5A2RK2Q6MVNMYOL3Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:13.883361725 (27m49.922934845s) [upload=399]: Next returned nil segment Sep 6 13:56:13.883376664 (27m49.922949784s) [upload=399]: Waiting for error group managing segments... Sep 6 13:56:14.173140084 (27m50.212713204s) [noise=true , address=45.91.116.10:4083 , piece_id=X3WQ622EA6UWZUWRGZ3745BHZZPIXRJFFNZIW4NMAF6G56ZIRVZA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.220496311 (27m50.260069431s) [noise=true , address=45.91.116.5:4298 , piece_id=GG6YIMNE6SRNYEWZSY2U4KY27YEYUH2XUXNFHXRGOKWIJD2BDL4Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.255626334 (27m50.295199454s) [noise=true , address=77.90.31.19:4843 , piece_id=A2PW54R4F2HRUCX7TQSEUD7JPX65FQE5MLA5A2RK2Q6MVNMYOL3Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.263389889 (27m50.302963009s) [noise=true , address=77.90.31.12:4590 , piece_id=QA7A6MUVOUPSUK74PGWAIDSK6PKCKSDWFWDCTENJ4273WOWWWRWA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.287035495 (27m50.326608615s) [noise=true , address=77.90.31.19:4841 , piece_id=QGSCJFC7LLNVWVEH2G7OBYMCSCPDMFNB74OHB75725Z3T7R76OUA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.292975913 (27m50.332549043s) [noise=true , address=45.91.116.8:4045 , piece_id=FT3O2Q34T34CUEW2RYTCBQYLXMFFXMJJG4K3CTHCSTVVICGNOAOQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.303591796 (27m50.343164916s) [noise=true , address=45.91.116.4:4938 , piece_id=DJVQ5EZUJVPJNQYXKLTVPJ2WEFYTUI42AALAODWQJSGVPYGXCUXQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.342732192 (27m50.382305312s) [noise=true , address=45.91.116.8:4047 , piece_id=GLNQE64DXUGQQDOPBKUEVTQUPYKIY36OXYY3AT2VU4CVD32WNWOA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.384729616 (27m50.424302726s) [noise=true , address=77.90.31.16:4734 , piece_id=5E4RJK2DXZKN554GPNRWMDBQHGNYNV6XYUOF6PKTFVVX4TSG5XHA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.411143620 (27m50.45071674s) [noise=true , address=77.90.31.15:4698 , piece_id=FKDMDFVF6T7IDOCWEND7JIQYHEA7QAZ7QT3BVMB6HB4INDP4BACA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.447251863 (27m50.486824984s) [noise=true , address=45.91.116.6:4973 , piece_id=CPLW4X7AI7FLX3SVEBWCWL35THGBVFU37NRX4P2LC36Z3ZXYVULA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.456897671 (27m50.496470791s) [noise=true , address=77.90.31.10:4518 , piece_id=PXX3UUN25OUY3PMLSOY2RYI5V3S2XC7D3YDFLKLLAT77NIR6B5CQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.490628396 (27m50.530201516s) [noise=true , address=77.90.31.17:4772 , piece_id=ES2I34T7P4MBDFYVDIPCT4WHEZ7CLMYGEWSEA642KJ3DPS7Z2Q5Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.490655378 (27m50.530228498s) [seg_pos={0 0}, upload=399]: Segment reached optimal threshold of 13 pieces. Sep 6 13:56:14.490779695 (27m50.530352805s) [noise=true , address=45.91.116.14:4189 , piece_id=BNDUXGDW2EPEPJXZANLOOZG2LUIOVU3EI5MCG6QZLLEA52HFTD4A, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF Sep 6 13:56:14.490926587 (27m50.530499707s) [noise=true , address=45.91.116.11:4118 , piece_id=S5M4G5WK6PKHVHBYZGIW7YNZEQUPRLY4VPWANNAT7247K5DFQWVA, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 13:56:14.491022340 (27m50.53059545s) [noise=true , address=77.90.31.15:4699 , piece_id=HO3DW4BYBCXHNLZBIDJQQ6AE7AYTDE6NXVE3YVZU3MJWFOFVWFZA, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: read tcp 188.166.225.170:45032->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 13:56:14.491037539 (27m50.530610659s) [seg_pos={0 0}, upload=399]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:56:14.491121650 (27m50.53069477s) [upload=399]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:56:14.491126880 (27m50.5307s ) [upload=399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:56:14.796736622 (27m50.836309742s) [upload=399]: Done uploading segments... Sep 6 13:56:14.796756030 (27m50.83632916s) [upload=399]: Upload finished. err: <nil> Duration: 10.221408991s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 14 out of 60 is failed (threshold=3) [........................XX......XX........XXX...XX...XX..XXX] duration: 30.001848892s FAIL (2025-09-06 13:56:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:56:11.784228618 (27m47.823801738s) [upload=399]: Starting upload Sep 6 13:56:11.784284775 (27m47.823857895s) [upload=399]: Done starting upload Sep 6 13:56:11.784435193 (27m47.824008313s) [upload=399]: Uploading segments... Sep 6 13:56:11.784479398 (27m47.824052508s) [upload=399]: Got next segment. Inline: false Sep 6 13:56:11.784498935 (27m47.824072055s) [upload=399]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:56:11.784503313 (27m47.824076433s) [upload=399]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:56:12.025562526 (27m48.065135646s) [seg_pos={0 0}, upload=399]: Begin upload segment... Sep 6 13:56:12.025614836 (27m48.065187956s) [seg_pos={0 0}, upload=399]: Done begin upload segment. Sep 6 13:56:12.025649193 (27m48.065222313s) [noise=true , address=77.90.31.10:4518 , piece_id=PXX3UUN25OUY3PMLSOY2RYI5V3S2XC7D3YDFLKLLAT77NIR6B5CQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026146965 (27m48.065720075s) [noise=true , address=77.90.31.15:4698 , piece_id=FKDMDFVF6T7IDOCWEND7JIQYHEA7QAZ7QT3BVMB6HB4INDP4BACA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026488648 (27m48.066061768s) [noise=true , address=45.91.116.8:4047 , piece_id=GLNQE64DXUGQQDOPBKUEVTQUPYKIY36OXYY3AT2VU4CVD32WNWOA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.026825463 (27m48.066398583s) [noise=true , address=45.91.116.11:4118 , piece_id=S5M4G5WK6PKHVHBYZGIW7YNZEQUPRLY4VPWANNAT7247K5DFQWVA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.027414019 (27m48.066987139s) [noise=true , address=77.90.31.12:4590 , piece_id=QA7A6MUVOUPSUK74PGWAIDSK6PKCKSDWFWDCTENJ4273WOWWWRWA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028280808 (27m48.067853928s) [noise=true , address=45.91.116.5:4298 , piece_id=GG6YIMNE6SRNYEWZSY2U4KY27YEYUH2XUXNFHXRGOKWIJD2BDL4Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028527510 (27m48.06810063s) [noise=true , address=45.91.116.14:4189 , piece_id=BNDUXGDW2EPEPJXZANLOOZG2LUIOVU3EI5MCG6QZLLEA52HFTD4A, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.028762650 (27m48.06833577s) [noise=true , address=77.90.31.19:4841 , piece_id=QGSCJFC7LLNVWVEH2G7OBYMCSCPDMFNB74OHB75725Z3T7R76OUA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029007118 (27m48.068580238s) [noise=true , address=45.91.116.4:4938 , piece_id=DJVQ5EZUJVPJNQYXKLTVPJ2WEFYTUI42AALAODWQJSGVPYGXCUXQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029314787 (27m48.068887917s) [noise=true , address=77.90.31.17:4772 , piece_id=ES2I34T7P4MBDFYVDIPCT4WHEZ7CLMYGEWSEA642KJ3DPS7Z2Q5Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029568472 (27m48.069141592s) [noise=true , address=77.90.31.16:4734 , piece_id=5E4RJK2DXZKN554GPNRWMDBQHGNYNV6XYUOF6PKTFVVX4TSG5XHA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.029846073 (27m48.069419183s) [noise=true , address=77.90.31.15:4699 , piece_id=HO3DW4BYBCXHNLZBIDJQQ6AE7AYTDE6NXVE3YVZU3MJWFOFVWFZA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030161848 (27m48.069734968s) [noise=true , address=45.91.116.8:4045 , piece_id=FT3O2Q34T34CUEW2RYTCBQYLXMFFXMJJG4K3CTHCSTVVICGNOAOQ, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030396566 (27m48.069969686s) [noise=true , address=45.91.116.6:4973 , piece_id=CPLW4X7AI7FLX3SVEBWCWL35THGBVFU37NRX4P2LC36Z3ZXYVULA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.030629903 (27m48.070203023s) [noise=true , address=45.91.116.10:4083 , piece_id=X3WQ622EA6UWZUWRGZ3745BHZZPIXRJFFNZIW4NMAF6G56ZIRVZA, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:12.031224090 (27m48.07079721s) [noise=true , address=77.90.31.19:4843 , piece_id=A2PW54R4F2HRUCX7TQSEUD7JPX65FQE5MLA5A2RK2Q6MVNMYOL3Q, seg_pos={0 0}, upload=399]: Uploading piece... Sep 6 13:56:13.883361725 (27m49.922934845s) [upload=399]: Next returned nil segment Sep 6 13:56:13.883376664 (27m49.922949784s) [upload=399]: Waiting for error group managing segments... Sep 6 13:56:14.173140084 (27m50.212713204s) [noise=true , address=45.91.116.10:4083 , piece_id=X3WQ622EA6UWZUWRGZ3745BHZZPIXRJFFNZIW4NMAF6G56ZIRVZA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.220496311 (27m50.260069431s) [noise=true , address=45.91.116.5:4298 , piece_id=GG6YIMNE6SRNYEWZSY2U4KY27YEYUH2XUXNFHXRGOKWIJD2BDL4Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.255626334 (27m50.295199454s) [noise=true , address=77.90.31.19:4843 , piece_id=A2PW54R4F2HRUCX7TQSEUD7JPX65FQE5MLA5A2RK2Q6MVNMYOL3Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.263389889 (27m50.302963009s) [noise=true , address=77.90.31.12:4590 , piece_id=QA7A6MUVOUPSUK74PGWAIDSK6PKCKSDWFWDCTENJ4273WOWWWRWA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.287035495 (27m50.326608615s) [noise=true , address=77.90.31.19:4841 , piece_id=QGSCJFC7LLNVWVEH2G7OBYMCSCPDMFNB74OHB75725Z3T7R76OUA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.292975913 (27m50.332549043s) [noise=true , address=45.91.116.8:4045 , piece_id=FT3O2Q34T34CUEW2RYTCBQYLXMFFXMJJG4K3CTHCSTVVICGNOAOQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.303591796 (27m50.343164916s) [noise=true , address=45.91.116.4:4938 , piece_id=DJVQ5EZUJVPJNQYXKLTVPJ2WEFYTUI42AALAODWQJSGVPYGXCUXQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.342732192 (27m50.382305312s) [noise=true , address=45.91.116.8:4047 , piece_id=GLNQE64DXUGQQDOPBKUEVTQUPYKIY36OXYY3AT2VU4CVD32WNWOA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.384729616 (27m50.424302726s) [noise=true , address=77.90.31.16:4734 , piece_id=5E4RJK2DXZKN554GPNRWMDBQHGNYNV6XYUOF6PKTFVVX4TSG5XHA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.411143620 (27m50.45071674s) [noise=true , address=77.90.31.15:4698 , piece_id=FKDMDFVF6T7IDOCWEND7JIQYHEA7QAZ7QT3BVMB6HB4INDP4BACA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.447251863 (27m50.486824984s) [noise=true , address=45.91.116.6:4973 , piece_id=CPLW4X7AI7FLX3SVEBWCWL35THGBVFU37NRX4P2LC36Z3ZXYVULA, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.456897671 (27m50.496470791s) [noise=true , address=77.90.31.10:4518 , piece_id=PXX3UUN25OUY3PMLSOY2RYI5V3S2XC7D3YDFLKLLAT77NIR6B5CQ, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.490628396 (27m50.530201516s) [noise=true , address=77.90.31.17:4772 , piece_id=ES2I34T7P4MBDFYVDIPCT4WHEZ7CLMYGEWSEA642KJ3DPS7Z2Q5Q, seg_pos={0 0}, upload=399]: Done uploading piece. err: <nil> Sep 6 13:56:14.490655378 (27m50.530228498s) [seg_pos={0 0}, upload=399]: Segment reached optimal threshold of 13 pieces. Sep 6 13:56:14.490779695 (27m50.530352805s) [noise=true , address=45.91.116.14:4189 , piece_id=BNDUXGDW2EPEPJXZANLOOZG2LUIOVU3EI5MCG6QZLLEA52HFTD4A, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JPMPojBpksikw8rtjnpFkz6WrRXWKqTJb1XE6eJpAAVnXtP6N): protocol: expected piece hash; context canceled; EOF Sep 6 13:56:14.490926587 (27m50.530499707s) [noise=true , address=45.91.116.11:4118 , piece_id=S5M4G5WK6PKHVHBYZGIW7YNZEQUPRLY4VPWANNAT7247K5DFQWVA, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ZDTj5nit2Gd5eVLYoeY8MeuMbvCHwAMKdzhVMx1j2QKTq3x7p): protocol: expected piece hash; context canceled; EOF Sep 6 13:56:14.491022340 (27m50.53059545s) [noise=true , address=77.90.31.15:4699 , piece_id=HO3DW4BYBCXHNLZBIDJQQ6AE7AYTDE6NXVE3YVZU3MJWFOFVWFZA, seg_pos={0 0}, upload=399]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: read tcp 188.166.225.170:45032->77.90.31.15:4699: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 6 13:56:14.491037539 (27m50.530610659s) [seg_pos={0 0}, upload=399]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:56:14.491121650 (27m50.53069477s) [upload=399]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:56:14.491126880 (27m50.5307s ) [upload=399]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:56:14.796736622 (27m50.836309742s) [upload=399]: Done uploading segments... Sep 6 13:56:14.796756030 (27m50.83632916s) [upload=399]: Upload finished. err: <nil> Duration: 30.001865924s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 13 out of 60 is failed (threshold=3) [.........................XX......XX........XXX...XX...XX..XX] duration: 30.002151671s FAIL (2025-09-06 13:55:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:55:14.055410447 (26m50.094983557s) [upload=383]: Starting upload Sep 6 13:55:14.055468067 (26m50.095041187s) [upload=383]: Done starting upload Sep 6 13:55:14.055545916 (26m50.095119036s) [upload=383]: Uploading segments... Sep 6 13:55:14.055723405 (26m50.095296525s) [upload=383]: Got next segment. Inline: false Sep 6 13:55:14.055817946 (26m50.095391066s) [upload=383]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:55:14.055823527 (26m50.095396697s) [upload=383]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:55:14.359714290 (26m50.39928741s) [seg_pos={0 0}, upload=383]: Begin upload segment... Sep 6 13:55:14.359761801 (26m50.399334921s) [seg_pos={0 0}, upload=383]: Done begin upload segment. Sep 6 13:55:14.359783894 (26m50.399357014s) [noise=true , address=77.90.31.17:4771 , piece_id=6B2V4PBH5IVB62D6FI6MYYWFSA6JD6RRZYRMRI2DKB2SE2BNIZYA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.359826875 (26m50.399399995s) [noise=true , address=77.90.31.16:4735 , piece_id=GDJO5U6BYD7J6ZHWNQO43UZKK6FLD554NE7SUQFOR6Y3V5IMH7MQ, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.360248813 (26m50.399821933s) [noise=true , address=45.91.116.8:4045 , piece_id=4TV4OGARTYQFXIKYHZZFBJHD2MEZPKMCOIXB27GTWLO44GKYD5YA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.360288949 (26m50.39986207s) [noise=true , address=45.91.116.4:4937 , piece_id=XJ4JYJCG7FDBZ5T4B633R5TMSMFHFAWZ2AEWH427KU5ZBIZEGLJA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.360584345 (26m50.400157465s) [noise=true , address=77.90.31.10:4517 , piece_id=TE7NLBDKTRDCBWKRV7B27BAEXJD4LYBDKOYNDZFX6TX4NWOOXALQ, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.360598662 (26m50.400171782s) [noise=true , address=77.90.31.18:4807 , piece_id=5UYLG77UTUY7U5CPG6QCGHPIQFCQJ5VB3WYZ6VQH3VOAOTJ6PJPQ, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.360939474 (26m50.400512595s) [noise=true , address=45.91.116.14:4189 , piece_id=HGM6G6MIWCVINZMLQ32O5625TQJICQIBZLC4FN65YHGARX2772XA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361018225 (26m50.400591345s) [noise=true , address=45.91.116.12:4227 , piece_id=LI3YARWG6MBLV3TUO3ZU5H24Q3V3HJ3I3UCTIIJUV4KOZ5YZL5RA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361274555 (26m50.400847665s) [noise=true , address=77.90.31.10:4520 , piece_id=2DEDBY36DQ2S54COJ5KUG5ZP45RJ2LQM5XLNK6WBDI7IZC75PV5A, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361310545 (26m50.400883665s) [noise=true , address=45.91.116.7:4010 , piece_id=Y5PPWVGKQSKBMYG2Q26B6L4TRZU755GZ36KNQWNANM2RMEAMNIWA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361574610 (26m50.40114773s) [noise=true , address=77.90.31.21:4877 , piece_id=3Q6MBCKAFAISLMR63VUN3U4VUL3RJ2QCTP3QNORBTJRTLKFKDJIA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361597654 (26m50.401170774s) [noise=true , address=45.91.116.4:4938 , piece_id=T5I6XPLONHX7H7FKSAX2A5CU4ZLBIQ2SNT7CXGMGJJIDGQLYGX7Q, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361840098 (26m50.401413218s) [noise=true , address=45.91.116.13:4155 , piece_id=KZNAXKY6YFVU2V6HTIE5WXZ24HOIWGPYEHIUHJ5N2VNW7R6WV5GA, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.361935209 (26m50.401508329s) [noise=true , address=45.91.116.9:4263 , piece_id=MXMIQXKZTH4ZZPFVCD2BJCY4WUY77LUSR66VG5SJEPNRVHI2YGSQ, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.362085848 (26m50.401658968s) [noise=true , address=77.90.31.12:4589 , piece_id=63ODF2VKMHB5DJOEKEHPZODQZX6AR2XZJ4INEW4O343PWHE2I4MQ, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:14.362434275 (26m50.402007395s) [noise=true , address=77.90.31.13:4626 , piece_id=FQJDJ23BCPO3RWTG7GNPGT4EB3AOY5BAN5R4XUP2RV7DZXSSCU5Q, seg_pos={0 0}, upload=383]: Uploading piece... Sep 6 13:55:16.204392822 (26m52.243965942s) [upload=383]: Next returned nil segment Sep 6 13:55:16.204416047 (26m52.243989167s) [upload=383]: Waiting for error group managing segments... Sep 6 13:55:16.494683190 (26m52.53425631s) [noise=true , address=77.90.31.10:4520 , piece_id=2DEDBY36DQ2S54COJ5KUG5ZP45RJ2LQM5XLNK6WBDI7IZC75PV5A, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.507438348 (26m52.547011468s) [noise=true , address=77.90.31.13:4626 , piece_id=FQJDJ23BCPO3RWTG7GNPGT4EB3AOY5BAN5R4XUP2RV7DZXSSCU5Q, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.541172770 (26m52.58074589s) [noise=true , address=77.90.31.17:4771 , piece_id=6B2V4PBH5IVB62D6FI6MYYWFSA6JD6RRZYRMRI2DKB2SE2BNIZYA, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.547432488 (26m52.587005608s) [noise=true , address=45.91.116.13:4155 , piece_id=KZNAXKY6YFVU2V6HTIE5WXZ24HOIWGPYEHIUHJ5N2VNW7R6WV5GA, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.552477541 (26m52.592050661s) [noise=true , address=45.91.116.8:4045 , piece_id=4TV4OGARTYQFXIKYHZZFBJHD2MEZPKMCOIXB27GTWLO44GKYD5YA, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.578611219 (26m52.618184339s) [noise=true , address=45.91.116.14:4189 , piece_id=HGM6G6MIWCVINZMLQ32O5625TQJICQIBZLC4FN65YHGARX2772XA, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.591790247 (26m52.631363367s) [noise=true , address=77.90.31.16:4735 , piece_id=GDJO5U6BYD7J6ZHWNQO43UZKK6FLD554NE7SUQFOR6Y3V5IMH7MQ, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.610935773 (26m52.650508883s) [noise=true , address=45.91.116.9:4263 , piece_id=MXMIQXKZTH4ZZPFVCD2BJCY4WUY77LUSR66VG5SJEPNRVHI2YGSQ, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.611060752 (26m52.650633862s) [noise=true , address=77.90.31.18:4807 , piece_id=5UYLG77UTUY7U5CPG6QCGHPIQFCQJ5VB3WYZ6VQH3VOAOTJ6PJPQ, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.641222708 (26m52.680795828s) [noise=true , address=45.91.116.4:4938 , piece_id=T5I6XPLONHX7H7FKSAX2A5CU4ZLBIQ2SNT7CXGMGJJIDGQLYGX7Q, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.693565497 (26m52.733138617s) [noise=true , address=77.90.31.12:4589 , piece_id=63ODF2VKMHB5DJOEKEHPZODQZX6AR2XZJ4INEW4O343PWHE2I4MQ, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.744031184 (26m52.783604294s) [noise=true , address=77.90.31.21:4877 , piece_id=3Q6MBCKAFAISLMR63VUN3U4VUL3RJ2QCTP3QNORBTJRTLKFKDJIA, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.800268536 (26m52.839841646s) [noise=true , address=77.90.31.10:4517 , piece_id=TE7NLBDKTRDCBWKRV7B27BAEXJD4LYBDKOYNDZFX6TX4NWOOXALQ, seg_pos={0 0}, upload=383]: Done uploading piece. err: <nil> Sep 6 13:55:16.800293704 (26m52.839866824s) [seg_pos={0 0}, upload=383]: Segment reached optimal threshold of 13 pieces. Sep 6 13:55:16.800389127 (26m52.839962237s) [noise=true , address=45.91.116.7:4010 , piece_id=Y5PPWVGKQSKBMYG2Q26B6L4TRZU755GZ36KNQWNANM2RMEAMNIWA, seg_pos={0 0}, upload=383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w): protocol: expected piece hash; context canceled; EOF Sep 6 13:55:16.800553562 (26m52.840126672s) [noise=true , address=45.91.116.4:4937 , piece_id=XJ4JYJCG7FDBZ5T4B633R5TMSMFHFAWZ2AEWH427KU5ZBIZEGLJA, seg_pos={0 0}, upload=383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ry1jh9FXr14g7WU4bfgJH4qw71kmTugE5qbqEUZmf7oVZYRkq): protocol: write tcp 188.166.225.170:57592->45.91.116.4:4937: use of closed network connection; write tcp 188.166.225.170:57592->45.91.116.4:4937: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57592->45.91.116.4:4937: use of closed network connection Sep 6 13:55:16.800610280 (26m52.84018342s) [noise=true , address=45.91.116.12:4227 , piece_id=LI3YARWG6MBLV3TUO3ZU5H24Q3V3HJ3I3UCTIIJUV4KOZ5YZL5RA, seg_pos={0 0}, upload=383]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: write tcp 188.166.225.170:38968->45.91.116.12:4227: use of closed network connection; write tcp 188.166.225.170:38968->45.91.116.12:4227: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:38968->45.91.116.12:4227: use of closed network connection Sep 6 13:55:16.800673561 (26m52.840246681s) [seg_pos={0 0}, upload=383]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:55:16.800786647 (26m52.840360208s) [upload=383]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:55:16.800792860 (26m52.84036598s) [upload=383]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:55:17.133228273 (26m53.172801393s) [upload=383]: Done uploading segments... Sep 6 13:55:17.133246969 (26m53.172820089s) [upload=383]: Upload finished. err: <nil> Duration: 30.002194513s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 12 out of 60 is failed (threshold=3) [..........................XX......XX........XXX...XX...XX..X] duration: 30.0019057s FAIL (2025-09-06 13:54:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:54:24.178369762 (26m0.217942882s) [upload=372]: Starting upload Sep 6 13:54:24.178445718 (26m0.218018828s) [upload=372]: Done starting upload Sep 6 13:54:24.178599332 (26m0.218172452s) [upload=372]: Uploading segments... Sep 6 13:54:24.178693191 (26m0.218266311s) [upload=372]: Got next segment. Inline: false Sep 6 13:54:24.178722217 (26m0.218295337s) [upload=372]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:54:24.178729982 (26m0.218303102s) [upload=372]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:54:24.457963683 (26m0.497536793s) [seg_pos={0 0}, upload=372]: Begin upload segment... Sep 6 13:54:24.458009060 (26m0.49758218s) [seg_pos={0 0}, upload=372]: Done begin upload segment. Sep 6 13:54:24.458027275 (26m0.497600395s) [noise=true , address=77.90.31.17:4772 , piece_id=JHBHPBL6W2FZ7NGRLZWA6JRTE2USJEMZZBOPOGPONUAKR7NKT7VA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.458034709 (26m0.497607829s) [noise=true , address=45.91.116.4:4938 , piece_id=VRUME2GEZP7QIKTDXZDURQABRAGZ3CA6SPSRVUFQH7VFBMJDFH7Q, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.458394036 (26m0.497967156s) [noise=true , address=45.91.116.6:4975 , piece_id=5J5DRT7VFY5QFL2YI5FVONRGNA4ISEWFPSNJS5G74IAR5WY7YLVA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.458763884 (26m0.498337005s) [noise=true , address=45.91.116.7:4010 , piece_id=5VD2PWS32Y2YYXKTEGFFIMEBVKLWFODHRJJKDVVQYU524YDKL3GA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.459063738 (26m0.498636848s) [noise=true , address=77.90.31.10:4517 , piece_id=FJ2MQP6CX7Z5MSSB5D2SZPELUYSWFZC3TIRP6BSD24W7ETC7FO7A, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.459392157 (26m0.498965287s) [noise=true , address=45.91.116.13:4155 , piece_id=6EJXB7FKCL2Y6QXOTJV5SEYM2I3V3OQS23NSFQFD5VIPTTNZKHUA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.459646984 (26m0.499220104s) [noise=true , address=45.91.116.9:4263 , piece_id=WVNHHXN74RZBNN3YTCPUHCBHKYPSLHRQQUOKUYIYOM2E3ARVYUDA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.459935336 (26m0.499508456s) [noise=true , address=45.91.116.14:4189 , piece_id=T5QXZSQKEXNVF5J423Y5GYY6QXS4POQAHXSCAY422OV3JV7WJOTA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.460202006 (26m0.499775126s) [noise=true , address=77.90.31.16:4734 , piece_id=K36URQKX5YFA6WBRBZX6GFWGTSTYEDCLUOBRXGNSIRTFV7SNRGDQ, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.460482473 (26m0.500055593s) [noise=true , address=45.91.116.14:4190 , piece_id=BXOKJDAJJ4ICIJMRASULPZR4IL422FPF6TQ462KTMIV7QTW3ZLFA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.460711080 (26m0.5002842s) [noise=true , address=77.90.31.21:4877 , piece_id=4KQK4AUEJ6YTLZTCBZUEDYUIQM4PRXSU4EYTU5OTWKLGOHNJ6MJQ, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.460928697 (26m0.500501817s) [noise=true , address=45.91.116.11:4117 , piece_id=Y3X3KPGZNNCFLZEUNP3UXZXD4IGN742MDUAPU6HLILCQ2YHB646Q, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.461154569 (26m0.500727679s) [noise=true , address=77.90.31.15:4699 , piece_id=OXT7NWNCZNE266T4DPCARJBR3XUZZS6SFGJOFVDOUUXY7GVP563A, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.461392374 (26m0.500965504s) [noise=true , address=77.90.31.17:4770 , piece_id=GIKH2KQED6Y4QTH2POMTMJBLL6GE7YHQ5HIF7NHIPGCQTYB7Y7XQ, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.461702818 (26m0.501275938s) [noise=true , address=45.91.116.6:4974 , piece_id=XYLY5NJEWIV6Y7P5TXLXGQKZ2KVFLTIH3BMC6N5ZURQBOLKJSZTQ, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:24.461914964 (26m0.501488074s) [noise=true , address=77.90.31.19:4842 , piece_id=L34M2A35BULBKKU5LYWTHVGVGZSXJXGA3LZF5YGRVBZKPODGCETA, seg_pos={0 0}, upload=372]: Uploading piece... Sep 6 13:54:26.317534982 (26m2.357108102s) [upload=372]: Next returned nil segment Sep 6 13:54:26.317552266 (26m2.357125376s) [upload=372]: Waiting for error group managing segments... Sep 6 13:54:26.611155082 (26m2.650728192s) [noise=true , address=77.90.31.16:4734 , piece_id=K36URQKX5YFA6WBRBZX6GFWGTSTYEDCLUOBRXGNSIRTFV7SNRGDQ, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.639190699 (26m2.678763819s) [noise=true , address=45.91.116.14:4189 , piece_id=T5QXZSQKEXNVF5J423Y5GYY6QXS4POQAHXSCAY422OV3JV7WJOTA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.657759141 (26m2.697332872s) [noise=true , address=45.91.116.6:4975 , piece_id=5J5DRT7VFY5QFL2YI5FVONRGNA4ISEWFPSNJS5G74IAR5WY7YLVA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.679770110 (26m2.71934323s) [noise=true , address=77.90.31.21:4877 , piece_id=4KQK4AUEJ6YTLZTCBZUEDYUIQM4PRXSU4EYTU5OTWKLGOHNJ6MJQ, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.684137977 (26m2.723711097s) [noise=true , address=45.91.116.14:4190 , piece_id=BXOKJDAJJ4ICIJMRASULPZR4IL422FPF6TQ462KTMIV7QTW3ZLFA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.689773391 (26m2.729346521s) [noise=true , address=45.91.116.11:4117 , piece_id=Y3X3KPGZNNCFLZEUNP3UXZXD4IGN742MDUAPU6HLILCQ2YHB646Q, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.691642547 (26m2.731215667s) [noise=true , address=77.90.31.17:4772 , piece_id=JHBHPBL6W2FZ7NGRLZWA6JRTE2USJEMZZBOPOGPONUAKR7NKT7VA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.694237393 (26m2.733810503s) [noise=true , address=45.91.116.9:4263 , piece_id=WVNHHXN74RZBNN3YTCPUHCBHKYPSLHRQQUOKUYIYOM2E3ARVYUDA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.726462416 (26m2.766035536s) [noise=true , address=45.91.116.13:4155 , piece_id=6EJXB7FKCL2Y6QXOTJV5SEYM2I3V3OQS23NSFQFD5VIPTTNZKHUA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.766585884 (26m2.806159004s) [noise=true , address=77.90.31.10:4517 , piece_id=FJ2MQP6CX7Z5MSSB5D2SZPELUYSWFZC3TIRP6BSD24W7ETC7FO7A, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.767854701 (26m2.807427821s) [noise=true , address=45.91.116.6:4974 , piece_id=XYLY5NJEWIV6Y7P5TXLXGQKZ2KVFLTIH3BMC6N5ZURQBOLKJSZTQ, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.774789872 (26m2.814362982s) [noise=true , address=45.91.116.7:4010 , piece_id=5VD2PWS32Y2YYXKTEGFFIMEBVKLWFODHRJJKDVVQYU524YDKL3GA, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.786572999 (26m2.826146119s) [noise=true , address=77.90.31.17:4770 , piece_id=GIKH2KQED6Y4QTH2POMTMJBLL6GE7YHQ5HIF7NHIPGCQTYB7Y7XQ, seg_pos={0 0}, upload=372]: Done uploading piece. err: <nil> Sep 6 13:54:26.786595352 (26m2.826168472s) [seg_pos={0 0}, upload=372]: Segment reached optimal threshold of 13 pieces. Sep 6 13:54:26.786731593 (26m2.826304713s) [noise=true , address=77.90.31.19:4842 , piece_id=L34M2A35BULBKKU5LYWTHVGVGZSXJXGA3LZF5YGRVBZKPODGCETA, seg_pos={0 0}, upload=372]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1YHj4FooV9PriUJCKWPDkd47nApVSUBCJwYXWADZodPLbuUZi3): protocol: expected piece hash; context canceled; EOF Sep 6 13:54:26.786789864 (26m2.826362984s) [noise=true , address=45.91.116.4:4938 , piece_id=VRUME2GEZP7QIKTDXZDURQABRAGZ3CA6SPSRVUFQH7VFBMJDFH7Q, seg_pos={0 0}, upload=372]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12M8xyCtkASb2kvUGac4p1RfGL9t623D81fse7we15eLFAH6rbW): protocol: expected piece hash; context canceled; EOF Sep 6 13:54:26.786908912 (26m2.826482032s) [noise=true , address=77.90.31.15:4699 , piece_id=OXT7NWNCZNE266T4DPCARJBR3XUZZS6SFGJOFVDOUUXY7GVP563A, seg_pos={0 0}, upload=372]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M): protocol: write tcp 188.166.225.170:57864->77.90.31.15:4699: use of closed network connection; write tcp 188.166.225.170:57864->77.90.31.15:4699: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57864->77.90.31.15:4699: use of closed network connection Sep 6 13:54:26.786939029 (26m2.826512149s) [seg_pos={0 0}, upload=372]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:54:26.787015165 (26m2.826588285s) [upload=372]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:54:26.787021447 (26m2.826594577s) [upload=372]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:54:27.101355013 (26m3.140928204s) [upload=372]: Done uploading segments... Sep 6 13:54:27.101381654 (26m3.140954804s) [upload=372]: Upload finished. err: <nil> Duration: 30.001927651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [...........................XX......XX........XXX...XX...XX..] duration: 24.916028479s FAIL (2025-09-06 13:53:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:51:24.477456159 (23m0.517029279s) [upload=331]: Starting upload Sep 6 13:51:24.477514490 (23m0.51708761s) [upload=331]: Done starting upload Sep 6 13:51:24.477615213 (23m0.517188343s) [upload=331]: Uploading segments... Sep 6 13:51:24.478272391 (23m0.517845511s) [upload=331]: Got next segment. Inline: false Sep 6 13:51:24.478356522 (23m0.517929642s) [upload=331]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:51:24.478384024 (23m0.517957144s) [upload=331]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:51:24.728893751 (23m0.768466871s) [seg_pos={0 0}, upload=331]: Begin upload segment... Sep 6 13:51:24.728931073 (23m0.768504193s) [seg_pos={0 0}, upload=331]: Done begin upload segment. Sep 6 13:51:24.728952434 (23m0.768525554s) [noise=true , address=77.90.31.16:4734 , piece_id=OBX4Z3EML3UMPZDPI53KMG3RZS7HP4CBUIGJQ4OCAAXYTGN27JRA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.728981419 (23m0.768554539s) [noise=true , address=45.91.116.13:4155 , piece_id=2FG5QXQNLR6GPFSI733TN342FGWSPVFF3E4LCAVOMULZF4GTCG4Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729393247 (23m0.768966367s) [noise=true , address=45.91.116.14:4189 , piece_id=7XOWL3GJEA2KYJXTW4AR3GJDDOUV36M7LRSVNWUUR5LHBC57CH3Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729518326 (23m0.769091446s) [noise=true , address=45.91.116.4:4937 , piece_id=D7QIAJLLY7OKWKVJUYLMPETY3ON5VOJWXBD3ZO7AKFLZJ7XFLXJQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729740261 (23m0.769313381s) [noise=true , address=45.91.116.4:4939 , piece_id=WXKWFF2F47OU2AURTVLTWIZ2TERAAIE5K6XGVBAFPAW4545AMN7A, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729888595 (23m0.769461715s) [noise=true , address=77.90.31.17:4771 , piece_id=GKANRKFQBPPTH4TGPSA7VIZT4ONGITDWTO6MJL43T4NKXWMKATOQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730092304 (23m0.769665424s) [noise=true , address=45.91.116.8:4045 , piece_id=3VMVZ2F2PKW2VEFHRDD4U6YGAXKJAOZ24BIC5UT3RZAP2K554JGQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730440230 (23m0.77001335s) [noise=true , address=45.91.116.14:4190 , piece_id=JBNWWOIWPZHC7BSDPHKHRQU46PJGE6KAVIFGOGTEC3R7U75KEY6Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730745043 (23m0.770318164s) [noise=true , address=77.90.31.13:4628 , piece_id=REGNB6QKHNOCDIGQIWC5CXN7CNQJ2EIBYNLM4OPTGW6DCK65FUIQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.731027113 (23m0.770600233s) [noise=true , address=77.90.31.10:4517 , piece_id=3FGYZPMJ4PHLSPFKJY2HTR5HOZ7AZMOXS7WIRZUN3GQA43VOX2IA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.732669056 (23m0.772242176s) [noise=true , address=45.91.116.12:4227 , piece_id=VHZNAPQRZOEIBHWDDYI36TVPEHBTVACJCZSISLJGCTN6XAD2XU3A, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733122583 (23m0.772695693s) [noise=true , address=45.91.116.6:4975 , piece_id=OOJMZDV6ZXDMOYNQABJR63SNMYIBS6OOL4GF3I5YEE23KSRVSLFQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733487873 (23m0.773060983s) [noise=true , address=77.90.31.10:4518 , piece_id=IWPTYR6RXACPJ2OHLPSHTJD3BBPYHQ5OL4MWWMZE7FMKFWVODNCQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733779450 (23m0.77335257s) [noise=true , address=45.91.116.11:4118 , piece_id=6O3Y5WAZPWPZBT76SUTHVX5NRPBLBHLZYT2IIYQDN4AVVWODCSBA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.734827195 (23m0.774400315s) [noise=true , address=45.91.116.5:4299 , piece_id=SYDJEVBMQOWJT74LN47J4PTTDWFEKQFSQIS46SXGKLGRQNTMHLVA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.735369322 (23m0.774942442s) [noise=true , address=77.90.31.12:4589 , piece_id=35XUZVVGOYYPLK7IKYDWERPNSJY2PGM6F2AYIB6IHWA7SJY6NTLQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:26.553144254 (23m2.592717364s) [upload=331]: Next returned nil segment Sep 6 13:51:26.553158390 (23m2.59273151s) [upload=331]: Waiting for error group managing segments... Sep 6 13:51:26.867673253 (23m2.907246373s) [noise=true , address=45.91.116.4:4937 , piece_id=D7QIAJLLY7OKWKVJUYLMPETY3ON5VOJWXBD3ZO7AKFLZJ7XFLXJQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.883097006 (23m2.922670126s) [noise=true , address=45.91.116.4:4939 , piece_id=WXKWFF2F47OU2AURTVLTWIZ2TERAAIE5K6XGVBAFPAW4545AMN7A, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.909594641 (23m2.949167761s) [noise=true , address=45.91.116.5:4299 , piece_id=SYDJEVBMQOWJT74LN47J4PTTDWFEKQFSQIS46SXGKLGRQNTMHLVA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.934022454 (23m2.973595574s) [noise=true , address=45.91.116.8:4045 , piece_id=3VMVZ2F2PKW2VEFHRDD4U6YGAXKJAOZ24BIC5UT3RZAP2K554JGQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.955376867 (23m2.994949997s) [noise=true , address=45.91.116.12:4227 , piece_id=VHZNAPQRZOEIBHWDDYI36TVPEHBTVACJCZSISLJGCTN6XAD2XU3A, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.965547168 (23m3.005120288s) [noise=true , address=45.91.116.14:4189 , piece_id=7XOWL3GJEA2KYJXTW4AR3GJDDOUV36M7LRSVNWUUR5LHBC57CH3Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.997475493 (23m3.037048613s) [noise=true , address=77.90.31.16:4734 , piece_id=OBX4Z3EML3UMPZDPI53KMG3RZS7HP4CBUIGJQ4OCAAXYTGN27JRA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.010327496 (23m3.049900616s) [noise=true , address=77.90.31.10:4517 , piece_id=3FGYZPMJ4PHLSPFKJY2HTR5HOZ7AZMOXS7WIRZUN3GQA43VOX2IA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.048754377 (23m3.088327497s) [noise=true , address=77.90.31.12:4589 , piece_id=35XUZVVGOYYPLK7IKYDWERPNSJY2PGM6F2AYIB6IHWA7SJY6NTLQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.082728627 (23m3.122301747s) [noise=true , address=77.90.31.17:4771 , piece_id=GKANRKFQBPPTH4TGPSA7VIZT4ONGITDWTO6MJL43T4NKXWMKATOQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.287833481 (23m3.327406601s) [noise=true , address=77.90.31.10:4518 , piece_id=IWPTYR6RXACPJ2OHLPSHTJD3BBPYHQ5OL4MWWMZE7FMKFWVODNCQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.370294903 (23m3.409868023s) [noise=true , address=45.91.116.11:4118 , piece_id=6O3Y5WAZPWPZBT76SUTHVX5NRPBLBHLZYT2IIYQDN4AVVWODCSBA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.832861397 (23m3.872434517s) [noise=true , address=77.90.31.13:4628 , piece_id=REGNB6QKHNOCDIGQIWC5CXN7CNQJ2EIBYNLM4OPTGW6DCK65FUIQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.832885944 (23m3.872459064s) [seg_pos={0 0}, upload=331]: Segment reached optimal threshold of 13 pieces. Sep 6 13:51:27.833060928 (23m3.872634048s) [noise=true , address=45.91.116.6:4975 , piece_id=OOJMZDV6ZXDMOYNQABJR63SNMYIBS6OOL4GF3I5YEE23KSRVSLFQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Sep 6 13:51:27.833088861 (23m3.872661981s) [noise=true , address=45.91.116.14:4190 , piece_id=JBNWWOIWPZHC7BSDPHKHRQU46PJGE6KAVIFGOGTEC3R7U75KEY6Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Sep 6 13:51:27.833143536 (23m3.872716656s) [noise=true , address=45.91.116.13:4155 , piece_id=2FG5QXQNLR6GPFSI733TN342FGWSPVFF3E4LCAVOMULZF4GTCG4Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection Sep 6 13:51:27.833161972 (23m3.872735082s) [seg_pos={0 0}, upload=331]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:51:27.833223359 (23m3.872796479s) [upload=331]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:51:27.833227296 (23m3.872800416s) [upload=331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:51:28.148779875 (23m4.188352995s) [upload=331]: Done uploading segments... Sep 6 13:51:28.148819902 (23m4.188393022s) [upload=331]: Upload finished. err: <nil> Duration: 24.916056101s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [............................XX......XX........XXX...XX...XX.] duration: 22.487960094s FAIL (2025-09-06 13:52:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:51:24.477456159 (23m0.517029279s) [upload=331]: Starting upload Sep 6 13:51:24.477514490 (23m0.51708761s) [upload=331]: Done starting upload Sep 6 13:51:24.477615213 (23m0.517188343s) [upload=331]: Uploading segments... Sep 6 13:51:24.478272391 (23m0.517845511s) [upload=331]: Got next segment. Inline: false Sep 6 13:51:24.478356522 (23m0.517929642s) [upload=331]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:51:24.478384024 (23m0.517957144s) [upload=331]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:51:24.728893751 (23m0.768466871s) [seg_pos={0 0}, upload=331]: Begin upload segment... Sep 6 13:51:24.728931073 (23m0.768504193s) [seg_pos={0 0}, upload=331]: Done begin upload segment. Sep 6 13:51:24.728952434 (23m0.768525554s) [noise=true , address=77.90.31.16:4734 , piece_id=OBX4Z3EML3UMPZDPI53KMG3RZS7HP4CBUIGJQ4OCAAXYTGN27JRA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.728981419 (23m0.768554539s) [noise=true , address=45.91.116.13:4155 , piece_id=2FG5QXQNLR6GPFSI733TN342FGWSPVFF3E4LCAVOMULZF4GTCG4Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729393247 (23m0.768966367s) [noise=true , address=45.91.116.14:4189 , piece_id=7XOWL3GJEA2KYJXTW4AR3GJDDOUV36M7LRSVNWUUR5LHBC57CH3Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729518326 (23m0.769091446s) [noise=true , address=45.91.116.4:4937 , piece_id=D7QIAJLLY7OKWKVJUYLMPETY3ON5VOJWXBD3ZO7AKFLZJ7XFLXJQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729740261 (23m0.769313381s) [noise=true , address=45.91.116.4:4939 , piece_id=WXKWFF2F47OU2AURTVLTWIZ2TERAAIE5K6XGVBAFPAW4545AMN7A, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729888595 (23m0.769461715s) [noise=true , address=77.90.31.17:4771 , piece_id=GKANRKFQBPPTH4TGPSA7VIZT4ONGITDWTO6MJL43T4NKXWMKATOQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730092304 (23m0.769665424s) [noise=true , address=45.91.116.8:4045 , piece_id=3VMVZ2F2PKW2VEFHRDD4U6YGAXKJAOZ24BIC5UT3RZAP2K554JGQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730440230 (23m0.77001335s) [noise=true , address=45.91.116.14:4190 , piece_id=JBNWWOIWPZHC7BSDPHKHRQU46PJGE6KAVIFGOGTEC3R7U75KEY6Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730745043 (23m0.770318164s) [noise=true , address=77.90.31.13:4628 , piece_id=REGNB6QKHNOCDIGQIWC5CXN7CNQJ2EIBYNLM4OPTGW6DCK65FUIQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.731027113 (23m0.770600233s) [noise=true , address=77.90.31.10:4517 , piece_id=3FGYZPMJ4PHLSPFKJY2HTR5HOZ7AZMOXS7WIRZUN3GQA43VOX2IA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.732669056 (23m0.772242176s) [noise=true , address=45.91.116.12:4227 , piece_id=VHZNAPQRZOEIBHWDDYI36TVPEHBTVACJCZSISLJGCTN6XAD2XU3A, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733122583 (23m0.772695693s) [noise=true , address=45.91.116.6:4975 , piece_id=OOJMZDV6ZXDMOYNQABJR63SNMYIBS6OOL4GF3I5YEE23KSRVSLFQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733487873 (23m0.773060983s) [noise=true , address=77.90.31.10:4518 , piece_id=IWPTYR6RXACPJ2OHLPSHTJD3BBPYHQ5OL4MWWMZE7FMKFWVODNCQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733779450 (23m0.77335257s) [noise=true , address=45.91.116.11:4118 , piece_id=6O3Y5WAZPWPZBT76SUTHVX5NRPBLBHLZYT2IIYQDN4AVVWODCSBA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.734827195 (23m0.774400315s) [noise=true , address=45.91.116.5:4299 , piece_id=SYDJEVBMQOWJT74LN47J4PTTDWFEKQFSQIS46SXGKLGRQNTMHLVA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.735369322 (23m0.774942442s) [noise=true , address=77.90.31.12:4589 , piece_id=35XUZVVGOYYPLK7IKYDWERPNSJY2PGM6F2AYIB6IHWA7SJY6NTLQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:26.553144254 (23m2.592717364s) [upload=331]: Next returned nil segment Sep 6 13:51:26.553158390 (23m2.59273151s) [upload=331]: Waiting for error group managing segments... Sep 6 13:51:26.867673253 (23m2.907246373s) [noise=true , address=45.91.116.4:4937 , piece_id=D7QIAJLLY7OKWKVJUYLMPETY3ON5VOJWXBD3ZO7AKFLZJ7XFLXJQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.883097006 (23m2.922670126s) [noise=true , address=45.91.116.4:4939 , piece_id=WXKWFF2F47OU2AURTVLTWIZ2TERAAIE5K6XGVBAFPAW4545AMN7A, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.909594641 (23m2.949167761s) [noise=true , address=45.91.116.5:4299 , piece_id=SYDJEVBMQOWJT74LN47J4PTTDWFEKQFSQIS46SXGKLGRQNTMHLVA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.934022454 (23m2.973595574s) [noise=true , address=45.91.116.8:4045 , piece_id=3VMVZ2F2PKW2VEFHRDD4U6YGAXKJAOZ24BIC5UT3RZAP2K554JGQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.955376867 (23m2.994949997s) [noise=true , address=45.91.116.12:4227 , piece_id=VHZNAPQRZOEIBHWDDYI36TVPEHBTVACJCZSISLJGCTN6XAD2XU3A, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.965547168 (23m3.005120288s) [noise=true , address=45.91.116.14:4189 , piece_id=7XOWL3GJEA2KYJXTW4AR3GJDDOUV36M7LRSVNWUUR5LHBC57CH3Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.997475493 (23m3.037048613s) [noise=true , address=77.90.31.16:4734 , piece_id=OBX4Z3EML3UMPZDPI53KMG3RZS7HP4CBUIGJQ4OCAAXYTGN27JRA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.010327496 (23m3.049900616s) [noise=true , address=77.90.31.10:4517 , piece_id=3FGYZPMJ4PHLSPFKJY2HTR5HOZ7AZMOXS7WIRZUN3GQA43VOX2IA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.048754377 (23m3.088327497s) [noise=true , address=77.90.31.12:4589 , piece_id=35XUZVVGOYYPLK7IKYDWERPNSJY2PGM6F2AYIB6IHWA7SJY6NTLQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.082728627 (23m3.122301747s) [noise=true , address=77.90.31.17:4771 , piece_id=GKANRKFQBPPTH4TGPSA7VIZT4ONGITDWTO6MJL43T4NKXWMKATOQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.287833481 (23m3.327406601s) [noise=true , address=77.90.31.10:4518 , piece_id=IWPTYR6RXACPJ2OHLPSHTJD3BBPYHQ5OL4MWWMZE7FMKFWVODNCQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.370294903 (23m3.409868023s) [noise=true , address=45.91.116.11:4118 , piece_id=6O3Y5WAZPWPZBT76SUTHVX5NRPBLBHLZYT2IIYQDN4AVVWODCSBA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.832861397 (23m3.872434517s) [noise=true , address=77.90.31.13:4628 , piece_id=REGNB6QKHNOCDIGQIWC5CXN7CNQJ2EIBYNLM4OPTGW6DCK65FUIQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.832885944 (23m3.872459064s) [seg_pos={0 0}, upload=331]: Segment reached optimal threshold of 13 pieces. Sep 6 13:51:27.833060928 (23m3.872634048s) [noise=true , address=45.91.116.6:4975 , piece_id=OOJMZDV6ZXDMOYNQABJR63SNMYIBS6OOL4GF3I5YEE23KSRVSLFQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Sep 6 13:51:27.833088861 (23m3.872661981s) [noise=true , address=45.91.116.14:4190 , piece_id=JBNWWOIWPZHC7BSDPHKHRQU46PJGE6KAVIFGOGTEC3R7U75KEY6Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Sep 6 13:51:27.833143536 (23m3.872716656s) [noise=true , address=45.91.116.13:4155 , piece_id=2FG5QXQNLR6GPFSI733TN342FGWSPVFF3E4LCAVOMULZF4GTCG4Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection Sep 6 13:51:27.833161972 (23m3.872735082s) [seg_pos={0 0}, upload=331]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:51:27.833223359 (23m3.872796479s) [upload=331]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:51:27.833227296 (23m3.872800416s) [upload=331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:51:28.148779875 (23m4.188352995s) [upload=331]: Done uploading segments... Sep 6 13:51:28.148819902 (23m4.188393022s) [upload=331]: Upload finished. err: <nil> Duration: 22.487987837s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 11 out of 60 is failed (threshold=3) [.............................XX......XX........XXX...XX...XX] duration: 30.00136735s FAIL (2025-09-06 13:51:53 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:51:24.477456159 (23m0.517029279s) [upload=331]: Starting upload Sep 6 13:51:24.477514490 (23m0.51708761s) [upload=331]: Done starting upload Sep 6 13:51:24.477615213 (23m0.517188343s) [upload=331]: Uploading segments... Sep 6 13:51:24.478272391 (23m0.517845511s) [upload=331]: Got next segment. Inline: false Sep 6 13:51:24.478356522 (23m0.517929642s) [upload=331]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:51:24.478384024 (23m0.517957144s) [upload=331]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:51:24.728893751 (23m0.768466871s) [seg_pos={0 0}, upload=331]: Begin upload segment... Sep 6 13:51:24.728931073 (23m0.768504193s) [seg_pos={0 0}, upload=331]: Done begin upload segment. Sep 6 13:51:24.728952434 (23m0.768525554s) [noise=true , address=77.90.31.16:4734 , piece_id=OBX4Z3EML3UMPZDPI53KMG3RZS7HP4CBUIGJQ4OCAAXYTGN27JRA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.728981419 (23m0.768554539s) [noise=true , address=45.91.116.13:4155 , piece_id=2FG5QXQNLR6GPFSI733TN342FGWSPVFF3E4LCAVOMULZF4GTCG4Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729393247 (23m0.768966367s) [noise=true , address=45.91.116.14:4189 , piece_id=7XOWL3GJEA2KYJXTW4AR3GJDDOUV36M7LRSVNWUUR5LHBC57CH3Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729518326 (23m0.769091446s) [noise=true , address=45.91.116.4:4937 , piece_id=D7QIAJLLY7OKWKVJUYLMPETY3ON5VOJWXBD3ZO7AKFLZJ7XFLXJQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729740261 (23m0.769313381s) [noise=true , address=45.91.116.4:4939 , piece_id=WXKWFF2F47OU2AURTVLTWIZ2TERAAIE5K6XGVBAFPAW4545AMN7A, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.729888595 (23m0.769461715s) [noise=true , address=77.90.31.17:4771 , piece_id=GKANRKFQBPPTH4TGPSA7VIZT4ONGITDWTO6MJL43T4NKXWMKATOQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730092304 (23m0.769665424s) [noise=true , address=45.91.116.8:4045 , piece_id=3VMVZ2F2PKW2VEFHRDD4U6YGAXKJAOZ24BIC5UT3RZAP2K554JGQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730440230 (23m0.77001335s) [noise=true , address=45.91.116.14:4190 , piece_id=JBNWWOIWPZHC7BSDPHKHRQU46PJGE6KAVIFGOGTEC3R7U75KEY6Q, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.730745043 (23m0.770318164s) [noise=true , address=77.90.31.13:4628 , piece_id=REGNB6QKHNOCDIGQIWC5CXN7CNQJ2EIBYNLM4OPTGW6DCK65FUIQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.731027113 (23m0.770600233s) [noise=true , address=77.90.31.10:4517 , piece_id=3FGYZPMJ4PHLSPFKJY2HTR5HOZ7AZMOXS7WIRZUN3GQA43VOX2IA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.732669056 (23m0.772242176s) [noise=true , address=45.91.116.12:4227 , piece_id=VHZNAPQRZOEIBHWDDYI36TVPEHBTVACJCZSISLJGCTN6XAD2XU3A, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733122583 (23m0.772695693s) [noise=true , address=45.91.116.6:4975 , piece_id=OOJMZDV6ZXDMOYNQABJR63SNMYIBS6OOL4GF3I5YEE23KSRVSLFQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733487873 (23m0.773060983s) [noise=true , address=77.90.31.10:4518 , piece_id=IWPTYR6RXACPJ2OHLPSHTJD3BBPYHQ5OL4MWWMZE7FMKFWVODNCQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.733779450 (23m0.77335257s) [noise=true , address=45.91.116.11:4118 , piece_id=6O3Y5WAZPWPZBT76SUTHVX5NRPBLBHLZYT2IIYQDN4AVVWODCSBA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.734827195 (23m0.774400315s) [noise=true , address=45.91.116.5:4299 , piece_id=SYDJEVBMQOWJT74LN47J4PTTDWFEKQFSQIS46SXGKLGRQNTMHLVA, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:24.735369322 (23m0.774942442s) [noise=true , address=77.90.31.12:4589 , piece_id=35XUZVVGOYYPLK7IKYDWERPNSJY2PGM6F2AYIB6IHWA7SJY6NTLQ, seg_pos={0 0}, upload=331]: Uploading piece... Sep 6 13:51:26.553144254 (23m2.592717364s) [upload=331]: Next returned nil segment Sep 6 13:51:26.553158390 (23m2.59273151s) [upload=331]: Waiting for error group managing segments... Sep 6 13:51:26.867673253 (23m2.907246373s) [noise=true , address=45.91.116.4:4937 , piece_id=D7QIAJLLY7OKWKVJUYLMPETY3ON5VOJWXBD3ZO7AKFLZJ7XFLXJQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.883097006 (23m2.922670126s) [noise=true , address=45.91.116.4:4939 , piece_id=WXKWFF2F47OU2AURTVLTWIZ2TERAAIE5K6XGVBAFPAW4545AMN7A, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.909594641 (23m2.949167761s) [noise=true , address=45.91.116.5:4299 , piece_id=SYDJEVBMQOWJT74LN47J4PTTDWFEKQFSQIS46SXGKLGRQNTMHLVA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.934022454 (23m2.973595574s) [noise=true , address=45.91.116.8:4045 , piece_id=3VMVZ2F2PKW2VEFHRDD4U6YGAXKJAOZ24BIC5UT3RZAP2K554JGQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.955376867 (23m2.994949997s) [noise=true , address=45.91.116.12:4227 , piece_id=VHZNAPQRZOEIBHWDDYI36TVPEHBTVACJCZSISLJGCTN6XAD2XU3A, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.965547168 (23m3.005120288s) [noise=true , address=45.91.116.14:4189 , piece_id=7XOWL3GJEA2KYJXTW4AR3GJDDOUV36M7LRSVNWUUR5LHBC57CH3Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:26.997475493 (23m3.037048613s) [noise=true , address=77.90.31.16:4734 , piece_id=OBX4Z3EML3UMPZDPI53KMG3RZS7HP4CBUIGJQ4OCAAXYTGN27JRA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.010327496 (23m3.049900616s) [noise=true , address=77.90.31.10:4517 , piece_id=3FGYZPMJ4PHLSPFKJY2HTR5HOZ7AZMOXS7WIRZUN3GQA43VOX2IA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.048754377 (23m3.088327497s) [noise=true , address=77.90.31.12:4589 , piece_id=35XUZVVGOYYPLK7IKYDWERPNSJY2PGM6F2AYIB6IHWA7SJY6NTLQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.082728627 (23m3.122301747s) [noise=true , address=77.90.31.17:4771 , piece_id=GKANRKFQBPPTH4TGPSA7VIZT4ONGITDWTO6MJL43T4NKXWMKATOQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.287833481 (23m3.327406601s) [noise=true , address=77.90.31.10:4518 , piece_id=IWPTYR6RXACPJ2OHLPSHTJD3BBPYHQ5OL4MWWMZE7FMKFWVODNCQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.370294903 (23m3.409868023s) [noise=true , address=45.91.116.11:4118 , piece_id=6O3Y5WAZPWPZBT76SUTHVX5NRPBLBHLZYT2IIYQDN4AVVWODCSBA, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.832861397 (23m3.872434517s) [noise=true , address=77.90.31.13:4628 , piece_id=REGNB6QKHNOCDIGQIWC5CXN7CNQJ2EIBYNLM4OPTGW6DCK65FUIQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: <nil> Sep 6 13:51:27.832885944 (23m3.872459064s) [seg_pos={0 0}, upload=331]: Segment reached optimal threshold of 13 pieces. Sep 6 13:51:27.833060928 (23m3.872634048s) [noise=true , address=45.91.116.6:4975 , piece_id=OOJMZDV6ZXDMOYNQABJR63SNMYIBS6OOL4GF3I5YEE23KSRVSLFQ, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12oUsZAf26PYH66pEfjgF9ydtxCoUYspEcsrSRmdhC63woA1hf7): protocol: expected piece hash; context canceled; EOF Sep 6 13:51:27.833088861 (23m3.872661981s) [noise=true , address=45.91.116.14:4190 , piece_id=JBNWWOIWPZHC7BSDPHKHRQU46PJGE6KAVIFGOGTEC3R7U75KEY6Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1JVvtSnmC6UL5tNSGeJg27797Z4T1DBsMLqVche429yRKvZZyG): protocol: expected piece hash; context canceled; EOF Sep 6 13:51:27.833143536 (23m3.872716656s) [noise=true , address=45.91.116.13:4155 , piece_id=2FG5QXQNLR6GPFSI733TN342FGWSPVFF3E4LCAVOMULZF4GTCG4Q, seg_pos={0 0}, upload=331]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12A74D73fZwkdrEvMvuHQKWjBsKEhWZcRcgix2ZNeWSZKkhWUuf): protocol: write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection; write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51188->45.91.116.13:4155: use of closed network connection Sep 6 13:51:27.833161972 (23m3.872735082s) [seg_pos={0 0}, upload=331]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:51:27.833223359 (23m3.872796479s) [upload=331]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:51:27.833227296 (23m3.872800416s) [upload=331]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:51:28.148779875 (23m4.188352995s) [upload=331]: Done uploading segments... Sep 6 13:51:28.148819902 (23m4.188393022s) [upload=331]: Upload finished. err: <nil> Duration: 30.001382339s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..............................XX......XX........XXX...XX...X] duration: 30.005348258s FAIL (2025-09-06 13:50:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:50:15.844557177 (21m51.884130297s) [upload=315]: Starting upload Sep 6 13:50:15.844605750 (21m51.88417887s) [upload=315]: Done starting upload Sep 6 13:50:15.844831703 (21m51.884404823s) [upload=315]: Uploading segments... Sep 6 13:50:15.844881508 (21m51.884454628s) [upload=315]: Got next segment. Inline: false Sep 6 13:50:15.844936133 (21m51.884509253s) [upload=315]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:50:15.844940661 (21m51.884513781s) [upload=315]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:50:16.121427195 (21m52.161000315s) [seg_pos={0 0}, upload=315]: Begin upload segment... Sep 6 13:50:16.121483272 (21m52.161056393s) [seg_pos={0 0}, upload=315]: Done begin upload segment. Sep 6 13:50:16.121535753 (21m52.161108873s) [noise=true , address=45.91.116.10:4082 , piece_id=JLCS2LEHQBC6N2CPKQ3GGY2FSEXGEPCXDPOQTVXXRKEROUMOQETQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.121966738 (21m52.161539858s) [noise=true , address=77.90.31.13:4628 , piece_id=BPBAZ73JP5TA3WYEXUCF3G5UR5STI5JEQO3FZAXPJYINUEVQVENA, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.122011162 (21m52.161584282s) [noise=true , address=45.91.116.5:4299 , piece_id=NJM3GNE4MZS724L3L5V76HF7RWWNNH6LAJCTEUUHABTHCW6P3LEQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.122361914 (21m52.161935034s) [noise=true , address=45.91.116.7:4009 , piece_id=4LJOVT5OCUXAB3MZHNVBRQWLQ7JP53BBJTAENWH2TJNWPG5JNCSQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.122491211 (21m52.162064331s) [noise=true , address=45.91.116.11:4119 , piece_id=P6GO6ZCHAGGNV56Q2V4DXQJOKNU4BL6FYH7JVIGQDHEDHA4FSFIQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.122657810 (21m52.16223093s) [noise=true , address=77.90.31.16:4734 , piece_id=H5R5NAR5D2Z3F66ZPXSZ4AU53NNJFDIF2LZMXJQEC6JVN4S7WK4A, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.122831803 (21m52.162404923s) [noise=true , address=45.91.116.8:4047 , piece_id=QQCNX3QDHPQVFAPITPB7KDO2QRTOIC2LWKKOQERMEZ5FKTO4R7IQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.123014112 (21m52.162587232s) [noise=true , address=45.91.116.4:4938 , piece_id=3SSHJHEGJTAGQNNWV36HSLBU6TNEHMGZ6IT7ZOHCCZGH3CR7EGBA, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.123133861 (21m52.162706981s) [noise=true , address=77.90.31.15:4698 , piece_id=S4VWP52FHJED2GCAO4CZDWCQGLCZV6BZ45S7HZILJTLS5D4TWKHA, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.123498890 (21m52.16307201s) [noise=true , address=45.91.116.9:4263 , piece_id=AH7QSUQMLVZ7GM4YZPO2BELCCELZERAXU4KNL47RG5IC4EEPRL7A, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.123544818 (21m52.163117938s) [noise=true , address=77.90.31.19:4843 , piece_id=HKXBR46NJVYVAXGEGFCPGFQLMVOKSUETHUKTLJS7QZIUYI2KMGSA, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.123829833 (21m52.163402953s) [noise=true , address=45.91.116.9:4262 , piece_id=R4DM3Y3GJI4LSRPJJIZ3C35K5VXKFZN4KOCP5DBSA63D2LBSUOVQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.123865150 (21m52.16343827s) [noise=true , address=45.91.116.12:4225 , piece_id=73WCZL2AYLNC6OCI2RNI6BA3JPOHUKJ75WTOQ23G7VKHB2JJJ3XA, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.124109858 (21m52.163682978s) [noise=true , address=45.91.116.13:4153 , piece_id=IDOAK3OANEMNYNBAXYJSYSB422NGKDDDJI254V46HYNTC2LE7HXQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.124456742 (21m52.164029863s) [noise=true , address=77.90.31.19:4842 , piece_id=X3M7CZJFHHKPJ3TVQXWJTV5UNRQE7DPCQULXP3JCFHQRPJ6LZB3Q, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:16.125864786 (21m52.165437906s) [noise=true , address=77.90.31.13:4626 , piece_id=TERL53SOQRD6Z2NK4A5UREQ2CW4VQ6O2GHUDCLWO7DQH5G7S3HJQ, seg_pos={0 0}, upload=315]: Uploading piece... Sep 6 13:50:17.938436201 (21m53.978009321s) [upload=315]: Next returned nil segment Sep 6 13:50:17.938451841 (21m53.978024961s) [upload=315]: Waiting for error group managing segments... Sep 6 13:50:18.237078821 (21m54.276651941s) [noise=true , address=45.91.116.9:4263 , piece_id=AH7QSUQMLVZ7GM4YZPO2BELCCELZERAXU4KNL47RG5IC4EEPRL7A, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.255925465 (21m54.295498585s) [noise=true , address=45.91.116.13:4153 , piece_id=IDOAK3OANEMNYNBAXYJSYSB422NGKDDDJI254V46HYNTC2LE7HXQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.265771145 (21m54.305344265s) [noise=true , address=45.91.116.7:4009 , piece_id=4LJOVT5OCUXAB3MZHNVBRQWLQ7JP53BBJTAENWH2TJNWPG5JNCSQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.276729404 (21m54.316302524s) [noise=true , address=77.90.31.13:4626 , piece_id=TERL53SOQRD6Z2NK4A5UREQ2CW4VQ6O2GHUDCLWO7DQH5G7S3HJQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.295798022 (21m54.335371152s) [noise=true , address=45.91.116.10:4082 , piece_id=JLCS2LEHQBC6N2CPKQ3GGY2FSEXGEPCXDPOQTVXXRKEROUMOQETQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.304064040 (21m54.34363716s) [noise=true , address=77.90.31.15:4698 , piece_id=S4VWP52FHJED2GCAO4CZDWCQGLCZV6BZ45S7HZILJTLS5D4TWKHA, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.305892799 (21m54.345465919s) [noise=true , address=45.91.116.8:4047 , piece_id=QQCNX3QDHPQVFAPITPB7KDO2QRTOIC2LWKKOQERMEZ5FKTO4R7IQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.328983334 (21m54.368556454s) [noise=true , address=77.90.31.19:4842 , piece_id=X3M7CZJFHHKPJ3TVQXWJTV5UNRQE7DPCQULXP3JCFHQRPJ6LZB3Q, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.343648184 (21m54.383221305s) [noise=true , address=77.90.31.16:4734 , piece_id=H5R5NAR5D2Z3F66ZPXSZ4AU53NNJFDIF2LZMXJQEC6JVN4S7WK4A, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.379962252 (21m54.419535393s) [noise=true , address=45.91.116.9:4262 , piece_id=R4DM3Y3GJI4LSRPJJIZ3C35K5VXKFZN4KOCP5DBSA63D2LBSUOVQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.400551681 (21m54.440124821s) [noise=true , address=77.90.31.19:4843 , piece_id=HKXBR46NJVYVAXGEGFCPGFQLMVOKSUETHUKTLJS7QZIUYI2KMGSA, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.403115808 (21m54.442688958s) [noise=true , address=45.91.116.12:4225 , piece_id=73WCZL2AYLNC6OCI2RNI6BA3JPOHUKJ75WTOQ23G7VKHB2JJJ3XA, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.404447375 (21m54.444020495s) [noise=true , address=45.91.116.4:4938 , piece_id=3SSHJHEGJTAGQNNWV36HSLBU6TNEHMGZ6IT7ZOHCCZGH3CR7EGBA, seg_pos={0 0}, upload=315]: Done uploading piece. err: <nil> Sep 6 13:50:18.404465621 (21m54.444038751s) [seg_pos={0 0}, upload=315]: Segment reached optimal threshold of 13 pieces. Sep 6 13:50:18.404660694 (21m54.444233814s) [noise=true , address=45.91.116.5:4299 , piece_id=NJM3GNE4MZS724L3L5V76HF7RWWNNH6LAJCTEUUHABTHCW6P3LEQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX): protocol: expected piece hash; context canceled; EOF Sep 6 13:50:18.404686593 (21m54.444259713s) [noise=true , address=77.90.31.13:4628 , piece_id=BPBAZ73JP5TA3WYEXUCF3G5UR5STI5JEQO3FZAXPJYINUEVQVENA, seg_pos={0 0}, upload=315]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Sep 6 13:50:18.404772417 (21m54.444345527s) [noise=true , address=45.91.116.11:4119 , piece_id=P6GO6ZCHAGGNV56Q2V4DXQJOKNU4BL6FYH7JVIGQDHEDHA4FSFIQ, seg_pos={0 0}, upload=315]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gAiCZHUi4y1iJcxzca4PEg7kosGQn45pCXhQAA9ss7yk6UThm): protocol: write tcp 188.166.225.170:37326->45.91.116.11:4119: use of closed network connection; write tcp 188.166.225.170:37326->45.91.116.11:4119: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37326->45.91.116.11:4119: use of closed network connection Sep 6 13:50:18.404799018 (21m54.444372138s) [seg_pos={0 0}, upload=315]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:50:18.404877980 (21m54.4444511s) [upload=315]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:50:18.404913958 (21m54.444487078s) [upload=315]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:50:18.737192181 (21m54.776765301s) [upload=315]: Done uploading segments... Sep 6 13:50:18.737210366 (21m54.776783487s) [upload=315]: Upload finished. err: <nil> Duration: 30.00535975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [...............................XX......XX........XXX...XX...] duration: 24.738434868s FAIL (2025-09-06 13:50:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:47:56.900534846 (19m32.940107956s) [upload=279]: Starting upload Sep 6 13:47:56.900593238 (19m32.940166358s) [upload=279]: Done starting upload Sep 6 13:47:56.900706826 (19m32.940279946s) [upload=279]: Uploading segments... Sep 6 13:47:56.900778022 (19m32.940351132s) [upload=279]: Got next segment. Inline: false Sep 6 13:47:56.900814442 (19m32.940387562s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:47:56.900819071 (19m32.940392181s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:47:57.178620220 (19m33.21819334s) [seg_pos={0 0}, upload=279]: Begin upload segment... Sep 6 13:47:57.178665638 (19m33.218238758s) [seg_pos={0 0}, upload=279]: Done begin upload segment. Sep 6 13:47:57.178685636 (19m33.218258756s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.178689073 (19m33.218262193s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179035555 (19m33.218608675s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179551142 (19m33.219124272s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179835326 (19m33.219408446s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180066809 (19m33.219639929s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180293462 (19m33.219866582s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180532430 (19m33.22010555s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180726661 (19m33.220299781s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180935711 (19m33.220508821s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181127268 (19m33.220700388s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181320668 (19m33.220893788s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181514609 (19m33.221087729s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181699613 (19m33.221272733s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181930054 (19m33.221503174s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.182131901 (19m33.221705021s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:59.030193906 (19m35.069767026s) [upload=279]: Next returned nil segment Sep 6 13:47:59.030210448 (19m35.069783568s) [upload=279]: Waiting for error group managing segments... Sep 6 13:47:59.331147539 (19m35.37072066s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.332314322 (19m35.371887442s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.346591452 (19m35.386164572s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.347440206 (19m35.387013366s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.350054378 (19m35.389627498s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.356078074 (19m35.395651194s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.357401397 (19m35.396974517s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.364381042 (19m35.403954162s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.406672338 (19m35.446245458s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.407149942 (19m35.446723062s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.415810264 (19m35.455383384s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.464956557 (19m35.504529697s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468114419 (19m35.507687539s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468131582 (19m35.507704702s) [seg_pos={0 0}, upload=279]: Segment reached optimal threshold of 13 pieces. Sep 6 13:47:59.468263364 (19m35.507836484s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468387642 (19m35.507960762s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468468066 (19m35.508041186s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468480800 (19m35.508053921s) [seg_pos={0 0}, upload=279]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:47:59.468550794 (19m35.508123914s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:47:59.468554791 (19m35.508127911s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:48:00.997718206 (19m37.037291327s) [upload=279]: Done uploading segments... Sep 6 13:48:00.997737434 (19m37.037310554s) [upload=279]: Upload finished. err: <nil> Duration: 24.738457301s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [................................XX......XX........XXX...XX..] duration: 17.100676994s FAIL (2025-09-06 13:49:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:47:56.900534846 (19m32.940107956s) [upload=279]: Starting upload Sep 6 13:47:56.900593238 (19m32.940166358s) [upload=279]: Done starting upload Sep 6 13:47:56.900706826 (19m32.940279946s) [upload=279]: Uploading segments... Sep 6 13:47:56.900778022 (19m32.940351132s) [upload=279]: Got next segment. Inline: false Sep 6 13:47:56.900814442 (19m32.940387562s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:47:56.900819071 (19m32.940392181s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:47:57.178620220 (19m33.21819334s) [seg_pos={0 0}, upload=279]: Begin upload segment... Sep 6 13:47:57.178665638 (19m33.218238758s) [seg_pos={0 0}, upload=279]: Done begin upload segment. Sep 6 13:47:57.178685636 (19m33.218258756s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.178689073 (19m33.218262193s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179035555 (19m33.218608675s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179551142 (19m33.219124272s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179835326 (19m33.219408446s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180066809 (19m33.219639929s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180293462 (19m33.219866582s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180532430 (19m33.22010555s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180726661 (19m33.220299781s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180935711 (19m33.220508821s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181127268 (19m33.220700388s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181320668 (19m33.220893788s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181514609 (19m33.221087729s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181699613 (19m33.221272733s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181930054 (19m33.221503174s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.182131901 (19m33.221705021s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:59.030193906 (19m35.069767026s) [upload=279]: Next returned nil segment Sep 6 13:47:59.030210448 (19m35.069783568s) [upload=279]: Waiting for error group managing segments... Sep 6 13:47:59.331147539 (19m35.37072066s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.332314322 (19m35.371887442s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.346591452 (19m35.386164572s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.347440206 (19m35.387013366s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.350054378 (19m35.389627498s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.356078074 (19m35.395651194s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.357401397 (19m35.396974517s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.364381042 (19m35.403954162s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.406672338 (19m35.446245458s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.407149942 (19m35.446723062s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.415810264 (19m35.455383384s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.464956557 (19m35.504529697s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468114419 (19m35.507687539s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468131582 (19m35.507704702s) [seg_pos={0 0}, upload=279]: Segment reached optimal threshold of 13 pieces. Sep 6 13:47:59.468263364 (19m35.507836484s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468387642 (19m35.507960762s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468468066 (19m35.508041186s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468480800 (19m35.508053921s) [seg_pos={0 0}, upload=279]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:47:59.468550794 (19m35.508123914s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:47:59.468554791 (19m35.508127911s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:48:00.997718206 (19m37.037291327s) [upload=279]: Done uploading segments... Sep 6 13:48:00.997737434 (19m37.037310554s) [upload=279]: Upload finished. err: <nil> Duration: 17.100702342s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.................................XX......XX........XXX...XX.] duration: 11.079079957s FAIL (2025-09-06 13:49:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:47:56.900534846 (19m32.940107956s) [upload=279]: Starting upload Sep 6 13:47:56.900593238 (19m32.940166358s) [upload=279]: Done starting upload Sep 6 13:47:56.900706826 (19m32.940279946s) [upload=279]: Uploading segments... Sep 6 13:47:56.900778022 (19m32.940351132s) [upload=279]: Got next segment. Inline: false Sep 6 13:47:56.900814442 (19m32.940387562s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:47:56.900819071 (19m32.940392181s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:47:57.178620220 (19m33.21819334s) [seg_pos={0 0}, upload=279]: Begin upload segment... Sep 6 13:47:57.178665638 (19m33.218238758s) [seg_pos={0 0}, upload=279]: Done begin upload segment. Sep 6 13:47:57.178685636 (19m33.218258756s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.178689073 (19m33.218262193s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179035555 (19m33.218608675s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179551142 (19m33.219124272s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179835326 (19m33.219408446s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180066809 (19m33.219639929s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180293462 (19m33.219866582s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180532430 (19m33.22010555s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180726661 (19m33.220299781s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180935711 (19m33.220508821s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181127268 (19m33.220700388s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181320668 (19m33.220893788s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181514609 (19m33.221087729s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181699613 (19m33.221272733s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181930054 (19m33.221503174s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.182131901 (19m33.221705021s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:59.030193906 (19m35.069767026s) [upload=279]: Next returned nil segment Sep 6 13:47:59.030210448 (19m35.069783568s) [upload=279]: Waiting for error group managing segments... Sep 6 13:47:59.331147539 (19m35.37072066s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.332314322 (19m35.371887442s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.346591452 (19m35.386164572s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.347440206 (19m35.387013366s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.350054378 (19m35.389627498s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.356078074 (19m35.395651194s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.357401397 (19m35.396974517s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.364381042 (19m35.403954162s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.406672338 (19m35.446245458s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.407149942 (19m35.446723062s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.415810264 (19m35.455383384s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.464956557 (19m35.504529697s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468114419 (19m35.507687539s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468131582 (19m35.507704702s) [seg_pos={0 0}, upload=279]: Segment reached optimal threshold of 13 pieces. Sep 6 13:47:59.468263364 (19m35.507836484s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468387642 (19m35.507960762s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468468066 (19m35.508041186s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468480800 (19m35.508053921s) [seg_pos={0 0}, upload=279]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:47:59.468550794 (19m35.508123914s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:47:59.468554791 (19m35.508127911s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:48:00.997718206 (19m37.037291327s) [upload=279]: Done uploading segments... Sep 6 13:48:00.997737434 (19m37.037310554s) [upload=279]: Upload finished. err: <nil> Duration: 11.079105164s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..................................XX......XX........XXX...XX] duration: 30.002105714s FAIL (2025-09-06 13:48:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:47:56.900534846 (19m32.940107956s) [upload=279]: Starting upload Sep 6 13:47:56.900593238 (19m32.940166358s) [upload=279]: Done starting upload Sep 6 13:47:56.900706826 (19m32.940279946s) [upload=279]: Uploading segments... Sep 6 13:47:56.900778022 (19m32.940351132s) [upload=279]: Got next segment. Inline: false Sep 6 13:47:56.900814442 (19m32.940387562s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:47:56.900819071 (19m32.940392181s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:47:57.178620220 (19m33.21819334s) [seg_pos={0 0}, upload=279]: Begin upload segment... Sep 6 13:47:57.178665638 (19m33.218238758s) [seg_pos={0 0}, upload=279]: Done begin upload segment. Sep 6 13:47:57.178685636 (19m33.218258756s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.178689073 (19m33.218262193s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179035555 (19m33.218608675s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179551142 (19m33.219124272s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.179835326 (19m33.219408446s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180066809 (19m33.219639929s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180293462 (19m33.219866582s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180532430 (19m33.22010555s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180726661 (19m33.220299781s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.180935711 (19m33.220508821s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181127268 (19m33.220700388s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181320668 (19m33.220893788s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181514609 (19m33.221087729s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181699613 (19m33.221272733s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.181930054 (19m33.221503174s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:57.182131901 (19m33.221705021s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Uploading piece... Sep 6 13:47:59.030193906 (19m35.069767026s) [upload=279]: Next returned nil segment Sep 6 13:47:59.030210448 (19m35.069783568s) [upload=279]: Waiting for error group managing segments... Sep 6 13:47:59.331147539 (19m35.37072066s) [noise=true , address=77.90.31.10:4520 , piece_id=X66EKYYEJN7VLRZVSPA5IB4WQBRLGNWBFEWHR6FD4RBHRNOQ7QFA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.332314322 (19m35.371887442s) [noise=true , address=45.91.116.6:4973 , piece_id=TS5MA3LG567W6XI4GFFA57GLZPVD6FSVJYWUUX6EIZPRRSNHJNBA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.346591452 (19m35.386164572s) [noise=true , address=45.91.116.9:4263 , piece_id=S6K54E3HQWXOTO4EMR6RVTKGA3QTFJOFFWEGTJVBQAWUK6FNSNGA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.347440206 (19m35.387013366s) [noise=true , address=77.90.31.13:4628 , piece_id=R5U3IVBTYPJX7UASH5CBA6RXDX233R25ZIWGZFEUTXBNSHU5HFCA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.350054378 (19m35.389627498s) [noise=true , address=45.91.116.14:4190 , piece_id=36ETXYM5W4VFOPACVUPZRVBFTXACMGWI4JKZ4FD4HKTFJCCAB4VA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.356078074 (19m35.395651194s) [noise=true , address=45.91.116.14:4189 , piece_id=6QBE2Y4U7CQXNXRMXPTAQP7ER53SHVPSRZAD5A4E36GAIQVUNMQA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.357401397 (19m35.396974517s) [noise=true , address=45.91.116.11:4119 , piece_id=UILLCBRG2BW6DEAK7PREZNSN6DONEUB7MJMQYX6YP3XUSZ2POFMQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.364381042 (19m35.403954162s) [noise=true , address=45.91.116.7:4009 , piece_id=F7I56SNTLHEFXR5AENCV4W6323UQ7SXYOO5DAHT3WHZSQTFMNYZQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.406672338 (19m35.446245458s) [noise=true , address=45.91.116.6:4975 , piece_id=MQMU5S5RVXJG74OAX7ATZJTYVGVIQ6SW6MDO2UZJGIM5A5XU2CVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.407149942 (19m35.446723062s) [noise=true , address=45.91.116.11:4117 , piece_id=RDBOGGC4HVKHI5FRABNWJGT6QDJPFASFWYIDELKQPAQH3SGIV2GA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.415810264 (19m35.455383384s) [noise=true , address=45.91.116.4:4938 , piece_id=BA3GGJCARLMY2X5PSBBTRYZ2YCIRJMRUUG5XE4CA7F4OMQNQ3EVA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.464956557 (19m35.504529697s) [noise=true , address=77.90.31.12:4592 , piece_id=W5ZUOEC2KCUZNKAOF3U7VTK4ULXYCPIKBB2W5MQRK6IST6HYGIJQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468114419 (19m35.507687539s) [noise=true , address=77.90.31.12:4590 , piece_id=VAWWF7UGROZGA3OQUIMKDLBJS5NES2O74IKZU4EVFHPWV3QAWYSA, seg_pos={0 0}, upload=279]: Done uploading piece. err: <nil> Sep 6 13:47:59.468131582 (19m35.507704702s) [seg_pos={0 0}, upload=279]: Segment reached optimal threshold of 13 pieces. Sep 6 13:47:59.468263364 (19m35.507836484s) [noise=true , address=77.90.31.15:4698 , piece_id=OKMGZHZ43F4A4VBP6EDCGCDADX2H635GSVIYVTSHQRS5FXLFUVSQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468387642 (19m35.507960762s) [noise=true , address=77.90.31.19:4841 , piece_id=55LE3GMP5CRSPDLN3GV3BRY434VTIDVKR7GUB3MTDURK7VTAWWRA, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468468066 (19m35.508041186s) [noise=true , address=45.91.116.8:4047 , piece_id=XQVAKPOEK5QWTVUKMZDWBBPXTSUS62NUGAVD5VOZ3TDU2TUO4VRQ, seg_pos={0 0}, upload=279]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12g3HzcNJXwLseiT8QVdE5yRJc91qtYsH9D7HGTiRVkiwnQ7sdN): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:59.468480800 (19m35.508053921s) [seg_pos={0 0}, upload=279]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:47:59.468550794 (19m35.508123914s) [upload=279]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:47:59.468554791 (19m35.508127911s) [upload=279]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:48:00.997718206 (19m37.037291327s) [upload=279]: Done uploading segments... Sep 6 13:48:00.997737434 (19m37.037310554s) [upload=279]: Upload finished. err: <nil> Duration: 30.00214588s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...................................XX......XX........XXX...X] duration: 30.002505399s FAIL (2025-09-06 13:47:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: context deadline exceeded ======LOGS====== Sep 6 13:47:23.085606005 (18m59.125179115s) [upload=273]: Starting upload Sep 6 13:47:23.085653224 (18m59.125226344s) [upload=273]: Done starting upload Sep 6 13:47:23.085731404 (18m59.125304524s) [upload=273]: Uploading segments... Sep 6 13:47:23.085894476 (18m59.125467596s) [upload=273]: Got next segment. Inline: false Sep 6 13:47:23.085927048 (18m59.125500168s) [upload=273]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 6 13:47:23.085931868 (18m59.125504978s) [upload=273]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 6 13:47:23.360815174 (18m59.400388294s) [seg_pos={0 0}, upload=273]: Begin upload segment... Sep 6 13:47:23.360859429 (18m59.40043255s) [seg_pos={0 0}, upload=273]: Done begin upload segment. Sep 6 13:47:23.360881151 (18m59.400454271s) [noise=true , address=45.91.116.12:4225 , piece_id=WRGEC5V7GHNPQX53FEUOK3LKNKYX3YEMYNMWCQ455G5ZKIFF3XAA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.360919424 (18m59.400492544s) [noise=true , address=45.91.116.14:4191 , piece_id=ETDGUYKAAQ4RROBQ4U5JSQYWS3HFURAAH4DB6MT6PAIOYSRBREHQ, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.361227583 (18m59.400800704s) [noise=true , address=45.91.116.11:4119 , piece_id=B24YFR2KOTDPLTXG4IF776GVWCOUGSCPVGDLQAUJQ2ARCLCOWCGA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.361394282 (18m59.400967412s) [noise=true , address=45.91.116.6:4975 , piece_id=ZZP6YQIDVB77RO5JQDC2VINQLAXV7T7KT43KXDURDK6FNP5M3YTQ, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.361561733 (18m59.401134853s) [noise=true , address=45.91.116.7:4010 , piece_id=UEOGOTOSWBLXILJWMJZA5HCSP3OXJQD4J3K4DNQZ6P4QIHUCVE7A, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.361726017 (18m59.401299137s) [noise=true , address=45.91.116.9:4263 , piece_id=LKYYBMKQ5FBGSBG25SIM2PEKLIKQTNIHCOAJMMEEG3JGPGNBDJCQ, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.361865004 (18m59.401438124s) [noise=true , address=45.91.116.11:4118 , piece_id=UYSW6EFUG4QFGRJ2WT4BYDWB7PFXWOPH2SLRCTXINI7LVOWPEKEQ, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.362227607 (18m59.401800727s) [noise=true , address=77.90.31.18:4807 , piece_id=FNQLQ7PAYH52X6MDI7BNS6TW6M4GDXVVU5K7HHITJGBVEQJYWD6A, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.363039541 (18m59.402612661s) [noise=true , address=77.90.31.16:4734 , piece_id=ENH6SLLW33ONFEKGEJHKPTPKVXAXTUNFKX54XQ4CEV7RJIS4S33Q, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.363452602 (18m59.403025722s) [noise=true , address=45.91.116.14:4189 , piece_id=Y2EZTQXPH56Z3VZNUHMFZVZUZBY7AHUYACQ2YOS3Y3TV75MDM3EA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.364386398 (18m59.403959508s) [noise=true , address=77.90.31.15:4698 , piece_id=JSD6HJFS4L3B36NNAYTLADGTEK4T42TSKDE6L3NPZOKVPK2PZHEA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.364635335 (18m59.404208455s) [noise=true , address=77.90.31.13:4626 , piece_id=IX2GUC6LWZLMNVZI2H7GV6GTQ52S3QAMAXY4SF5MIRN4Y6NUUGFA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.365653312 (18m59.405226432s) [noise=true , address=45.91.116.8:4047 , piece_id=UONF6CGLYYXF25YOQLHUTE466JKGTHVOV7GBV5GYQWYME3NTKRVA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.366109796 (18m59.405682916s) [noise=true , address=77.90.31.18:4806 , piece_id=6TXZUGG4UTMHN6JQ3MLXO466LKC6QOPXCNOILUNG7M4ONDPNDV7Q, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.366494181 (18m59.406067302s) [noise=true , address=77.90.31.14:4663 , piece_id=3H6YHU2NCNLJN4YOTYZWIJ6YTEBI7UNEKQXE5XLFWK6P4H7ZPQXA, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:23.366894458 (18m59.406467578s) [noise=true , address=45.91.116.12:4226 , piece_id=A4HDV2ZD7SXXEOW6MPNLQT6IUUXL5MGJNJZHX2S5X72APNNWSC7Q, seg_pos={0 0}, upload=273]: Uploading piece... Sep 6 13:47:25.213254547 (19m1.252827667s) [upload=273]: Next returned nil segment Sep 6 13:47:25.213275157 (19m1.252848277s) [upload=273]: Waiting for error group managing segments... Sep 6 13:47:25.525909300 (19m1.56548242s) [noise=true , address=45.91.116.14:4191 , piece_id=ETDGUYKAAQ4RROBQ4U5JSQYWS3HFURAAH4DB6MT6PAIOYSRBREHQ, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.537668381 (19m1.577241501s) [noise=true , address=45.91.116.7:4010 , piece_id=UEOGOTOSWBLXILJWMJZA5HCSP3OXJQD4J3K4DNQZ6P4QIHUCVE7A, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.540870809 (19m1.580443929s) [noise=true , address=77.90.31.14:4663 , piece_id=3H6YHU2NCNLJN4YOTYZWIJ6YTEBI7UNEKQXE5XLFWK6P4H7ZPQXA, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.541705076 (19m1.581278196s) [noise=true , address=45.91.116.11:4118 , piece_id=UYSW6EFUG4QFGRJ2WT4BYDWB7PFXWOPH2SLRCTXINI7LVOWPEKEQ, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.543762182 (19m1.583335302s) [noise=true , address=45.91.116.11:4119 , piece_id=B24YFR2KOTDPLTXG4IF776GVWCOUGSCPVGDLQAUJQ2ARCLCOWCGA, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.544990262 (19m1.584563382s) [noise=true , address=45.91.116.8:4047 , piece_id=UONF6CGLYYXF25YOQLHUTE466JKGTHVOV7GBV5GYQWYME3NTKRVA, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.595784729 (19m1.635357849s) [noise=true , address=45.91.116.6:4975 , piece_id=ZZP6YQIDVB77RO5JQDC2VINQLAXV7T7KT43KXDURDK6FNP5M3YTQ, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.597261004 (19m1.636834124s) [noise=true , address=45.91.116.14:4189 , piece_id=Y2EZTQXPH56Z3VZNUHMFZVZUZBY7AHUYACQ2YOS3Y3TV75MDM3EA, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.599377874 (19m1.638950994s) [noise=true , address=77.90.31.18:4806 , piece_id=6TXZUGG4UTMHN6JQ3MLXO466LKC6QOPXCNOILUNG7M4ONDPNDV7Q, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.619718276 (19m1.659291396s) [noise=true , address=45.91.116.12:4225 , piece_id=WRGEC5V7GHNPQX53FEUOK3LKNKYX3YEMYNMWCQ455G5ZKIFF3XAA, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.621490738 (19m1.661063868s) [noise=true , address=77.90.31.16:4734 , piece_id=ENH6SLLW33ONFEKGEJHKPTPKVXAXTUNFKX54XQ4CEV7RJIS4S33Q, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.679943669 (19m1.719516799s) [noise=true , address=45.91.116.12:4226 , piece_id=A4HDV2ZD7SXXEOW6MPNLQT6IUUXL5MGJNJZHX2S5X72APNNWSC7Q, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.689526165 (19m1.729099285s) [noise=true , address=45.91.116.9:4263 , piece_id=LKYYBMKQ5FBGSBG25SIM2PEKLIKQTNIHCOAJMMEEG3JGPGNBDJCQ, seg_pos={0 0}, upload=273]: Done uploading piece. err: <nil> Sep 6 13:47:25.689552596 (19m1.729125716s) [seg_pos={0 0}, upload=273]: Segment reached optimal threshold of 13 pieces. Sep 6 13:47:25.689677084 (19m1.729250204s) [noise=true , address=77.90.31.13:4626 , piece_id=IX2GUC6LWZLMNVZI2H7GV6GTQ52S3QAMAXY4SF5MIRN4Y6NUUGFA, seg_pos={0 0}, upload=273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:25.689753621 (19m1.729326741s) [noise=true , address=77.90.31.15:4698 , piece_id=JSD6HJFS4L3B36NNAYTLADGTEK4T42TSKDE6L3NPZOKVPK2PZHEA, seg_pos={0 0}, upload=273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj): protocol: write tcp 188.166.225.170:53790->77.90.31.15:4698: use of closed network connection; write tcp 188.166.225.170:53790->77.90.31.15:4698: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53790->77.90.31.15:4698: use of closed network connection Sep 6 13:47:25.689761355 (19m1.729334475s) [noise=true , address=77.90.31.18:4807 , piece_id=FNQLQ7PAYH52X6MDI7BNS6TW6M4GDXVVU5K7HHITJGBVEQJYWD6A, seg_pos={0 0}, upload=273]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jVDPs8JFrPp3swUR5wBoEHpii8BS9F5Hb5p3vUe6btGKHfYaS): protocol: expected piece hash; context canceled; EOF Sep 6 13:47:25.689775363 (19m1.729348483s) [seg_pos={0 0}, upload=273]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 6 13:47:25.689916152 (19m1.729489272s) [upload=273]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 6 13:47:25.689924417 (19m1.729497538s) [upload=273]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 6 13:47:26.013445474 (19m2.053018594s) [upload=273]: Done uploading segments... Sep 6 13:47:26.013467606 (19m2.053040736s) [upload=273]: Upload finished. err: <nil> Duration: 30.002523292s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)