Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............................................X.....XXXXXXXX.] duration: 6.822671436s FAIL (2025-10-07 06:48:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 06:47:44.427004749 (329h16m29.889097232s) [upload=289774]: Starting upload Oct 7 06:47:44.427081815 (329h16m29.889174338s) [upload=289774]: Done starting upload Oct 7 06:47:44.427196734 (329h16m29.889289207s) [upload=289774]: Uploading segments... Oct 7 06:47:44.427392006 (329h16m29.889484499s) [upload=289774]: Got next segment. Inline: false Oct 7 06:47:44.427453333 (329h16m29.889545816s) [upload=289774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:47:44.427461128 (329h16m29.889553611s) [upload=289774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:47:44.679422423 (329h16m30.141514907s) [seg_pos={0 0}, upload=289774]: Begin upload segment... Oct 7 06:47:44.679499991 (329h16m30.141592464s) [seg_pos={0 0}, upload=289774]: Done begin upload segment. Oct 7 06:47:44.679535719 (329h16m30.141628192s) [noise=true , address=77.90.31.13:4627 , piece_id=BPQWDISSUSEYF6OLZQQ24LDVBCGMK2L7RHDUCFH2IE77575KTTRQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.679576326 (329h16m30.141668809s) [noise=true , address=45.91.116.12:4226 , piece_id=GO67AETH2MWDZWNBF4YEQL4Y24DY4Y3OTOSXYLEMH7W7PHZJ74HQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.680136603 (329h16m30.142229076s) [noise=true , address=145.79.227.14:4840 , piece_id=7IUNQ2TUZBQN3PJLYMQOQZCP6F2EOSQ2EPVRCT4GORS2FYF5THJQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.680582883 (329h16m30.142675356s) [noise=true , address=145.79.227.7:4295 , piece_id=S4MJJNZAJBA6TJMU7VAQ3VIHI6MB3BNX75PVO7HYZ5ZSUTZJONNA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.681155123 (329h16m30.143247606s) [noise=true , address=77.90.31.18:4807 , piece_id=7CHCHKBKJTRCPVB5CFAU2CIA6ADZYI2JKRZHWIF2NJMDM6RPP4PQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.681697185 (329h16m30.143789668s) [noise=true , address=45.91.116.7:4009 , piece_id=HB63JKX6ZOLOI3QAF4A32JESY5JM7JXD2HFZLTPVM4VDBOUWWPZQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.682199712 (329h16m30.144292185s) [noise=true , address=77.90.31.19:4841 , piece_id=WGDIEYTPX463VHAKBP3XOYPFNHKIPNQMMMB65RBD2ZCTHSW2P5PA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.682312607 (329h16m30.14440508s) [noise=true , address=145.79.227.11:4730 , piece_id=LBPIIBNO6UHQXT2ITLJWEHXLILZTML7DGIKWE4QR7I4L65NK3KDA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.682892250 (329h16m30.144984733s) [noise=true , address=45.91.116.10:4082 , piece_id=FG5RHFBYC44EZSUEPW7SLBLLJIXH475RZVPJFKOI6EAP6KQLPPWA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.683201669 (329h16m30.145294212s) [noise=true , address=77.90.31.11:4555 , piece_id=RMTY2E4WTCAWSFOWIMYNERGYNYVQJRHX5UAZ2WWEN7C7F2GV5IWQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.683633582 (329h16m30.145726055s) [noise=true , address=145.79.227.5:4223 , piece_id=FFFNYU7FW6MUYOBIRRLPNGHCUQ2D332QEQ6UDTGRIPJKJKG3O6XA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.684020950 (329h16m30.146113423s) [noise=true , address=77.90.31.17:4772 , piece_id=66LIQTI7PJQDOJUUNOBOI6PJYNT2AXIOWWPW4N6ND4T7YMEAZQ2A, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.684468452 (329h16m30.146560935s) [noise=true , address=77.90.31.21:4877 , piece_id=DHC7WX6XYFNGVHHF242YSFV5JX65LUQAM3EXYJKIQA6BPHLG573Q, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.684637264 (329h16m30.146729737s) [noise=true , address=77.90.31.16:4733 , piece_id=5KOEQNWVMKUSIV452ICFXHXIDPNL5T3NPKC75ZJ477YWNXY7NW7Q, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.685314212 (329h16m30.147406685s) [noise=true , address=145.79.227.19:4077 , piece_id=EP3XMKDD243FXDOXYO2YZVETZPLTGRY6VKRLFVKKWMJPSF3VXHMQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.685774819 (329h16m30.147867302s) [noise=true , address=45.91.116.4:4939 , piece_id=JJS2LLE2PPQ4FQLABPLQGH54HUKKCG4YEN6LRY3U22MO5ZVJQEZQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:46.512353104 (329h16m31.974445587s) [upload=289774]: Next returned nil segment Oct 7 06:47:46.512385024 (329h16m31.974477507s) [upload=289774]: Waiting for error group managing segments... Oct 7 06:47:46.803294934 (329h16m32.265387417s) [noise=true , address=45.91.116.10:4082 , piece_id=FG5RHFBYC44EZSUEPW7SLBLLJIXH475RZVPJFKOI6EAP6KQLPPWA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.810262832 (329h16m32.272355315s) [noise=true , address=77.90.31.13:4627 , piece_id=BPQWDISSUSEYF6OLZQQ24LDVBCGMK2L7RHDUCFH2IE77575KTTRQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.850094288 (329h16m32.312186771s) [noise=true , address=77.90.31.17:4772 , piece_id=66LIQTI7PJQDOJUUNOBOI6PJYNT2AXIOWWPW4N6ND4T7YMEAZQ2A, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.856207348 (329h16m32.318299831s) [noise=true , address=45.91.116.4:4939 , piece_id=JJS2LLE2PPQ4FQLABPLQGH54HUKKCG4YEN6LRY3U22MO5ZVJQEZQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.858225681 (329h16m32.320318154s) [noise=true , address=45.91.116.7:4009 , piece_id=HB63JKX6ZOLOI3QAF4A32JESY5JM7JXD2HFZLTPVM4VDBOUWWPZQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.861973348 (329h16m32.324065821s) [noise=true , address=77.90.31.19:4841 , piece_id=WGDIEYTPX463VHAKBP3XOYPFNHKIPNQMMMB65RBD2ZCTHSW2P5PA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.879126239 (329h16m32.341218722s) [noise=true , address=45.91.116.12:4226 , piece_id=GO67AETH2MWDZWNBF4YEQL4Y24DY4Y3OTOSXYLEMH7W7PHZJ74HQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.950243829 (329h16m32.412336322s) [noise=true , address=77.90.31.11:4555 , piece_id=RMTY2E4WTCAWSFOWIMYNERGYNYVQJRHX5UAZ2WWEN7C7F2GV5IWQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.964048705 (329h16m32.426141178s) [noise=true , address=77.90.31.18:4807 , piece_id=7CHCHKBKJTRCPVB5CFAU2CIA6ADZYI2JKRZHWIF2NJMDM6RPP4PQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.971634310 (329h16m32.433726783s) [noise=true , address=77.90.31.21:4877 , piece_id=DHC7WX6XYFNGVHHF242YSFV5JX65LUQAM3EXYJKIQA6BPHLG573Q, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:47.060358254 (329h16m32.522450737s) [noise=true , address=77.90.31.16:4733 , piece_id=5KOEQNWVMKUSIV452ICFXHXIDPNL5T3NPKC75ZJ477YWNXY7NW7Q, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:48:13.715201245 (329h16m59.177293728s) [noise=true , address=145.79.227.19:4077 , piece_id=EP3XMKDD243FXDOXYO2YZVETZPLTGRY6VKRLFVKKWMJPSF3VXHMQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: read tcp 188.166.225.170:60670->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715223448 (329h16m59.177315941s) [noise=true , address=145.79.227.11:4730 , piece_id=LBPIIBNO6UHQXT2ITLJWEHXLILZTML7DGIKWE4QR7I4L65NK3KDA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: read tcp 188.166.225.170:60978->145.79.227.11:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715365428 (329h16m59.177457911s) [noise=true , address=145.79.227.5:4223 , piece_id=FFFNYU7FW6MUYOBIRRLPNGHCUQ2D332QEQ6UDTGRIPJKJKG3O6XA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: read tcp 188.166.225.170:39876->145.79.227.5:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715431945 (329h16m59.177524438s) [noise=true , address=145.79.227.7:4295 , piece_id=S4MJJNZAJBA6TJMU7VAQ3VIHI6MB3BNX75PVO7HYZ5ZSUTZJONNA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:42926->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715579115 (329h16m59.177671608s) [noise=true , address=145.79.227.14:4840 , piece_id=7IUNQ2TUZBQN3PJLYMQOQZCP6F2EOSQ2EPVRCT4GORS2FYF5THJQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:57200->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715614482 (329h16m59.177706965s) [seg_pos={0 0}, upload=289774]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 06:48:13.969982742 (329h16m59.432075235s) [upload=289774]: Done uploading segments... Oct 7 06:48:13.970024111 (329h16m59.432116604s) [upload=289774]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.822697305s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [..............................................X.....XXXXXXXX] duration: 30.257312714s FAIL (2025-10-07 06:48:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 06:47:44.427004749 (329h16m29.889097232s) [upload=289774]: Starting upload Oct 7 06:47:44.427081815 (329h16m29.889174338s) [upload=289774]: Done starting upload Oct 7 06:47:44.427196734 (329h16m29.889289207s) [upload=289774]: Uploading segments... Oct 7 06:47:44.427392006 (329h16m29.889484499s) [upload=289774]: Got next segment. Inline: false Oct 7 06:47:44.427453333 (329h16m29.889545816s) [upload=289774]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:47:44.427461128 (329h16m29.889553611s) [upload=289774]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:47:44.679422423 (329h16m30.141514907s) [seg_pos={0 0}, upload=289774]: Begin upload segment... Oct 7 06:47:44.679499991 (329h16m30.141592464s) [seg_pos={0 0}, upload=289774]: Done begin upload segment. Oct 7 06:47:44.679535719 (329h16m30.141628192s) [noise=true , address=77.90.31.13:4627 , piece_id=BPQWDISSUSEYF6OLZQQ24LDVBCGMK2L7RHDUCFH2IE77575KTTRQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.679576326 (329h16m30.141668809s) [noise=true , address=45.91.116.12:4226 , piece_id=GO67AETH2MWDZWNBF4YEQL4Y24DY4Y3OTOSXYLEMH7W7PHZJ74HQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.680136603 (329h16m30.142229076s) [noise=true , address=145.79.227.14:4840 , piece_id=7IUNQ2TUZBQN3PJLYMQOQZCP6F2EOSQ2EPVRCT4GORS2FYF5THJQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.680582883 (329h16m30.142675356s) [noise=true , address=145.79.227.7:4295 , piece_id=S4MJJNZAJBA6TJMU7VAQ3VIHI6MB3BNX75PVO7HYZ5ZSUTZJONNA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.681155123 (329h16m30.143247606s) [noise=true , address=77.90.31.18:4807 , piece_id=7CHCHKBKJTRCPVB5CFAU2CIA6ADZYI2JKRZHWIF2NJMDM6RPP4PQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.681697185 (329h16m30.143789668s) [noise=true , address=45.91.116.7:4009 , piece_id=HB63JKX6ZOLOI3QAF4A32JESY5JM7JXD2HFZLTPVM4VDBOUWWPZQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.682199712 (329h16m30.144292185s) [noise=true , address=77.90.31.19:4841 , piece_id=WGDIEYTPX463VHAKBP3XOYPFNHKIPNQMMMB65RBD2ZCTHSW2P5PA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.682312607 (329h16m30.14440508s) [noise=true , address=145.79.227.11:4730 , piece_id=LBPIIBNO6UHQXT2ITLJWEHXLILZTML7DGIKWE4QR7I4L65NK3KDA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.682892250 (329h16m30.144984733s) [noise=true , address=45.91.116.10:4082 , piece_id=FG5RHFBYC44EZSUEPW7SLBLLJIXH475RZVPJFKOI6EAP6KQLPPWA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.683201669 (329h16m30.145294212s) [noise=true , address=77.90.31.11:4555 , piece_id=RMTY2E4WTCAWSFOWIMYNERGYNYVQJRHX5UAZ2WWEN7C7F2GV5IWQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.683633582 (329h16m30.145726055s) [noise=true , address=145.79.227.5:4223 , piece_id=FFFNYU7FW6MUYOBIRRLPNGHCUQ2D332QEQ6UDTGRIPJKJKG3O6XA, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.684020950 (329h16m30.146113423s) [noise=true , address=77.90.31.17:4772 , piece_id=66LIQTI7PJQDOJUUNOBOI6PJYNT2AXIOWWPW4N6ND4T7YMEAZQ2A, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.684468452 (329h16m30.146560935s) [noise=true , address=77.90.31.21:4877 , piece_id=DHC7WX6XYFNGVHHF242YSFV5JX65LUQAM3EXYJKIQA6BPHLG573Q, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.684637264 (329h16m30.146729737s) [noise=true , address=77.90.31.16:4733 , piece_id=5KOEQNWVMKUSIV452ICFXHXIDPNL5T3NPKC75ZJ477YWNXY7NW7Q, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.685314212 (329h16m30.147406685s) [noise=true , address=145.79.227.19:4077 , piece_id=EP3XMKDD243FXDOXYO2YZVETZPLTGRY6VKRLFVKKWMJPSF3VXHMQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:44.685774819 (329h16m30.147867302s) [noise=true , address=45.91.116.4:4939 , piece_id=JJS2LLE2PPQ4FQLABPLQGH54HUKKCG4YEN6LRY3U22MO5ZVJQEZQ, seg_pos={0 0}, upload=289774]: Uploading piece... Oct 7 06:47:46.512353104 (329h16m31.974445587s) [upload=289774]: Next returned nil segment Oct 7 06:47:46.512385024 (329h16m31.974477507s) [upload=289774]: Waiting for error group managing segments... Oct 7 06:47:46.803294934 (329h16m32.265387417s) [noise=true , address=45.91.116.10:4082 , piece_id=FG5RHFBYC44EZSUEPW7SLBLLJIXH475RZVPJFKOI6EAP6KQLPPWA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.810262832 (329h16m32.272355315s) [noise=true , address=77.90.31.13:4627 , piece_id=BPQWDISSUSEYF6OLZQQ24LDVBCGMK2L7RHDUCFH2IE77575KTTRQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.850094288 (329h16m32.312186771s) [noise=true , address=77.90.31.17:4772 , piece_id=66LIQTI7PJQDOJUUNOBOI6PJYNT2AXIOWWPW4N6ND4T7YMEAZQ2A, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.856207348 (329h16m32.318299831s) [noise=true , address=45.91.116.4:4939 , piece_id=JJS2LLE2PPQ4FQLABPLQGH54HUKKCG4YEN6LRY3U22MO5ZVJQEZQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.858225681 (329h16m32.320318154s) [noise=true , address=45.91.116.7:4009 , piece_id=HB63JKX6ZOLOI3QAF4A32JESY5JM7JXD2HFZLTPVM4VDBOUWWPZQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.861973348 (329h16m32.324065821s) [noise=true , address=77.90.31.19:4841 , piece_id=WGDIEYTPX463VHAKBP3XOYPFNHKIPNQMMMB65RBD2ZCTHSW2P5PA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.879126239 (329h16m32.341218722s) [noise=true , address=45.91.116.12:4226 , piece_id=GO67AETH2MWDZWNBF4YEQL4Y24DY4Y3OTOSXYLEMH7W7PHZJ74HQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.950243829 (329h16m32.412336322s) [noise=true , address=77.90.31.11:4555 , piece_id=RMTY2E4WTCAWSFOWIMYNERGYNYVQJRHX5UAZ2WWEN7C7F2GV5IWQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.964048705 (329h16m32.426141178s) [noise=true , address=77.90.31.18:4807 , piece_id=7CHCHKBKJTRCPVB5CFAU2CIA6ADZYI2JKRZHWIF2NJMDM6RPP4PQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:46.971634310 (329h16m32.433726783s) [noise=true , address=77.90.31.21:4877 , piece_id=DHC7WX6XYFNGVHHF242YSFV5JX65LUQAM3EXYJKIQA6BPHLG573Q, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:47:47.060358254 (329h16m32.522450737s) [noise=true , address=77.90.31.16:4733 , piece_id=5KOEQNWVMKUSIV452ICFXHXIDPNL5T3NPKC75ZJ477YWNXY7NW7Q, seg_pos={0 0}, upload=289774]: Done uploading piece. err: <nil> Oct 7 06:48:13.715201245 (329h16m59.177293728s) [noise=true , address=145.79.227.19:4077 , piece_id=EP3XMKDD243FXDOXYO2YZVETZPLTGRY6VKRLFVKKWMJPSF3VXHMQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: read tcp 188.166.225.170:60670->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715223448 (329h16m59.177315941s) [noise=true , address=145.79.227.11:4730 , piece_id=LBPIIBNO6UHQXT2ITLJWEHXLILZTML7DGIKWE4QR7I4L65NK3KDA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: read tcp 188.166.225.170:60978->145.79.227.11:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715365428 (329h16m59.177457911s) [noise=true , address=145.79.227.5:4223 , piece_id=FFFNYU7FW6MUYOBIRRLPNGHCUQ2D332QEQ6UDTGRIPJKJKG3O6XA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: read tcp 188.166.225.170:39876->145.79.227.5:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715431945 (329h16m59.177524438s) [noise=true , address=145.79.227.7:4295 , piece_id=S4MJJNZAJBA6TJMU7VAQ3VIHI6MB3BNX75PVO7HYZ5ZSUTZJONNA, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:42926->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715579115 (329h16m59.177671608s) [noise=true , address=145.79.227.14:4840 , piece_id=7IUNQ2TUZBQN3PJLYMQOQZCP6F2EOSQ2EPVRCT4GORS2FYF5THJQ, seg_pos={0 0}, upload=289774]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:57200->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:48:13.715614482 (329h16m59.177706965s) [seg_pos={0 0}, upload=289774]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 06:48:13.969982742 (329h16m59.432075235s) [upload=289774]: Done uploading segments... Oct 7 06:48:13.970024111 (329h16m59.432116604s) [upload=289774]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.257341699s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................................X....X.XXX] duration: 1m0.246295528s FAIL (2025-10-07 06:47:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== MVGTHVY75ZLSY7UWAC5SQGYY3TTTW3RWTYOYSXHOEO6SZ4FKAA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705111464 (329h15m23.167203937s) [noise=true , address=77.90.31.12:4606 , piece_id=IXAIGCSWW7VY3HFK7CF7CNLNM3TASDDBWJSRTOHGEWVQAP7ADHBQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705487209 (329h15m23.167579692s) [noise=true , address=145.79.227.23:4552 , piece_id=GVG2QYK6ZLFUQ65P4X6GIEC7HAVLP7EI4EBGDX5KGFIAAMQPIZ2Q, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705497098 (329h15m23.167589571s) [noise=true , address=77.90.31.13:4629 , piece_id=NRRW6N2T6YCTRMDDB3ZSWEOLTKQI2F3GKJDEBVVF65KXE3WJHKLA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705904694 (329h15m23.167997177s) [noise=true , address=77.90.31.10:4531 , piece_id=ORCE3C4ICPSKCAGUKXEUSXIF2SEDZNRT3BUILJ4YUMP7QXLBHVCQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.705959458 (329h15m23.168051932s) [noise=true , address=45.91.116.9:4270 , piece_id=FJFEUAGU7L4OKYIBKYQTHIRFVYT7CN3HIDMIE6AXEWLECZLBEETQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706211489 (329h15m23.168303962s) [noise=true , address=45.91.116.7:4043 , piece_id=CETYJJ4N5ZLRZSAAR2AH7R5OMQ7HVF5RJTNZBDUSDAKNKYXZUQGA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706330585 (329h15m23.168423068s) [noise=true , address=145.79.227.22:4513 , piece_id=NMGWKE7YVNOIXEP4D32M3YLGHKPM45V5T3GFF5R5TLASNY7N4ADA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706491422 (329h15m23.168583895s) [noise=true , address=77.90.31.17:4804 , piece_id=LOWJXG2XRC3FVDBEVS266D7E5AFHAH6XF6AGGQUYCEWYHHZECJBQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706669350 (329h15m23.168761823s) [noise=true , address=77.90.31.11:4575 , piece_id=ESE4YTFCLCX34UFK6GLUGMWDOIZJRFD5ZJAHNRSGX5RSWR2MQLEA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706792574 (329h15m23.168885078s) [noise=true , address=77.90.31.21:4890 , piece_id=BFVG3TIT6Q7NDHLVOXRTJSDZ7STT3RGBLCXAAKCB5GOHO55MPN5A, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.706989410 (329h15m23.169081883s) [noise=true , address=77.90.31.16:4738 , piece_id=S3XFQKWU6B5FYUBQXLLK2ILOXHXRAYWI47MXGULEP53QQT2BZAOA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707090182 (329h15m23.169182655s) [noise=true , address=145.79.227.27:4701 , piece_id=IDEVO4UTDUO5X6DO2YQUYHHGRMEEL7A6QEIOCQZ6XMVYFJZMQQOQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707276636 (329h15m23.169369109s) [noise=true , address=45.91.116.8:4060 , piece_id=5AXATBD5ZHE3DDLE6BGXB3W7WVKRXYKM25BYR2TNU7RFGCRCJZSQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707371978 (329h15m23.169464451s) [noise=true , address=145.79.227.33:4894 , piece_id=DMSAEZYC5FQDZCDJIGD3JN7OPIAQLP6U5RGKZXRYNPZQRP5MLUUQ, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707549766 (329h15m23.16964225s) [noise=true , address=145.79.227.52:4454 , piece_id=NM44WSGL6ZFTCS3JZ7XGXWXBWEZBUS4BHHP4VNZRJXPBGPWDNYWA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.707631533 (329h15m23.169724006s) [noise=true , address=145.79.227.26:4670 , piece_id=SUXAM2K4NBOY5JU2VHCFY3SPSYWR2LC4KSBID56RXQ6DU2OUFTVA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:37.708072953 (329h15m23.170165436s) [noise=true , address=145.79.227.16:4928 , piece_id=EZZSSR4MKZ2PLU7XB7KRZM47FWCJXYLYTFIL6LICB57JCE7ZF7LA, seg_pos={0 0}, upload=289755]: Uploading piece... Oct 7 06:46:39.225707749 (329h15m24.687800222s) [upload=289755]: Next returned nil segment Oct 7 06:46:39.225732907 (329h15m24.68782539s) [upload=289755]: Waiting for error group managing segments... Oct 7 06:46:39.615721759 (329h15m25.077814242s) [noise=true , address=45.91.116.9:4270 , piece_id=FJFEUAGU7L4OKYIBKYQTHIRFVYT7CN3HIDMIE6AXEWLECZLBEETQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.643120521 (329h15m25.105213004s) [noise=true , address=77.90.31.21:4890 , piece_id=BFVG3TIT6Q7NDHLVOXRTJSDZ7STT3RGBLCXAAKCB5GOHO55MPN5A, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.679706697 (329h15m25.14179918s) [noise=true , address=77.90.31.11:4575 , piece_id=ESE4YTFCLCX34UFK6GLUGMWDOIZJRFD5ZJAHNRSGX5RSWR2MQLEA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.682246323 (329h15m25.144338806s) [noise=true , address=77.90.31.17:4804 , piece_id=LOWJXG2XRC3FVDBEVS266D7E5AFHAH6XF6AGGQUYCEWYHHZECJBQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.682520384 (329h15m25.144612857s) [noise=true , address=45.91.116.8:4060 , piece_id=5AXATBD5ZHE3DDLE6BGXB3W7WVKRXYKM25BYR2TNU7RFGCRCJZSQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.727919902 (329h15m25.190012375s) [noise=true , address=77.90.31.16:4738 , piece_id=S3XFQKWU6B5FYUBQXLLK2ILOXHXRAYWI47MXGULEP53QQT2BZAOA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.732209110 (329h15m25.194301593s) [noise=true , address=45.91.116.7:4043 , piece_id=CETYJJ4N5ZLRZSAAR2AH7R5OMQ7HVF5RJTNZBDUSDAKNKYXZUQGA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.746507747 (329h15m25.20860022s) [noise=true , address=77.90.31.10:4531 , piece_id=ORCE3C4ICPSKCAGUKXEUSXIF2SEDZNRT3BUILJ4YUMP7QXLBHVCQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.793149551 (329h15m25.255242044s) [noise=true , address=77.90.31.12:4606 , piece_id=IXAIGCSWW7VY3HFK7CF7CNLNM3TASDDBWJSRTOHGEWVQAP7ADHBQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:39.989452452 (329h15m25.451544926s) [noise=true , address=45.91.116.10:4089 , piece_id=6ZMVGTHVY75ZLSY7UWAC5SQGYY3TTTW3RWTYOYSXHOEO6SZ4FKAA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:46:40.810206582 (329h15m26.272299066s) [noise=true , address=77.90.31.13:4629 , piece_id=NRRW6N2T6YCTRMDDB3ZSWEOLTKQI2F3GKJDEBVVF65KXE3WJHKLA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: <nil> Oct 7 06:47:36.689555151 (329h16m22.151647634s) [noise=true , address=145.79.227.26:4670 , piece_id=SUXAM2K4NBOY5JU2VHCFY3SPSYWR2LC4KSBID56RXQ6DU2OUFTVA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1Tn26F2GhEKm5HP7P7SFYjyBsRt31ueACyGzXdRYoPwEsK7jqp, address:145.79.227.26:4670): protocol: rpc: dial tcp 145.79.227.26:4670: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.689594726 (329h16m22.151687219s) [noise=true , address=145.79.227.23:4552 , piece_id=GVG2QYK6ZLFUQ65P4X6GIEC7HAVLP7EI4EBGDX5KGFIAAMQPIZ2Q, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:12YmWSZZQgXm2gdZCahnyCdSzxYPAerzCfD38mWPmpTDyGYT823, address:145.79.227.23:4552): protocol: read tcp 188.166.225.170:47042->145.79.227.23:4552: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.689879608 (329h16m22.151972091s) [noise=true , address=145.79.227.52:4454 , piece_id=NM44WSGL6ZFTCS3JZ7XGXWXBWEZBUS4BHHP4VNZRJXPBGPWDNYWA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1UCkuYGdVYLoR8oc1jpCZCN2smNfVqFwC8a7q95fJb8haB3MY2, address:145.79.227.52:4454): protocol: read tcp 188.166.225.170:57220->145.79.227.52:4454: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690032350 (329h16m22.152124843s) [noise=true , address=145.79.227.27:4701 , piece_id=IDEVO4UTDUO5X6DO2YQUYHHGRMEEL7A6QEIOCQZ6XMVYFJZMQQOQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1weCZWAtZRRYdeuPE8TjnZB5fZVD4gz4qW6TxyyNz33wVfW9hQ, address:145.79.227.27:4701): protocol: read tcp 188.166.225.170:56690->145.79.227.27:4701: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690084148 (329h16m22.152176641s) [noise=true , address=145.79.227.16:4928 , piece_id=EZZSSR4MKZ2PLU7XB7KRZM47FWCJXYLYTFIL6LICB57JCE7ZF7LA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:1fKSEPa36EtKVhzFZ2tCsQVi62RynY5Bwup1kQE17UB1bswsHg, address:145.79.227.16:4928): protocol: read tcp 188.166.225.170:49436->145.79.227.16:4928: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690243111 (329h16m22.152335594s) [noise=true , address=145.79.227.33:4894 , piece_id=DMSAEZYC5FQDZCDJIGD3JN7OPIAQLP6U5RGKZXRYNPZQRP5MLUUQ, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:12KWNyruYhmSJVvgUF7fBpGf4H7eYmtAeEXToYjd2dcEPcj1E44, address:145.79.227.33:4894): protocol: read tcp 188.166.225.170:41880->145.79.227.33:4894: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690289158 (329h16m22.152381651s) [noise=true , address=145.79.227.22:4513 , piece_id=NMGWKE7YVNOIXEP4D32M3YLGHKPM45V5T3GFF5R5TLASNY7N4ADA, seg_pos={0 0}, upload=289755]: Done uploading piece. err: ecclient: upload failed (node:12BykEXT7spgn4jkPucFmPvjvgKPKi5fZgRAeMTbfecZ7jCNori, address:145.79.227.22:4513): protocol: read tcp 188.166.225.170:34444->145.79.227.22:4513: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:36.690322742 (329h16m22.152415225s) [seg_pos={0 0}, upload=289755]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Oct 7 06:47:36.934316798 (329h16m22.396409271s) [upload=289755]: Done uploading segments... Oct 7 06:47:36.934355181 (329h16m22.396447664s) [upload=289755]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.246310207s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [...............................................X.....XXXXXXX] duration: 30.240626792s FAIL (2025-10-07 06:47:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 06:47:01.782554651 (329h15m47.244647134s) [upload=289762]: Starting upload Oct 7 06:47:01.782652358 (329h15m47.244744871s) [upload=289762]: Done starting upload Oct 7 06:47:01.782886262 (329h15m47.244978745s) [upload=289762]: Uploading segments... Oct 7 06:47:01.782967187 (329h15m47.24505967s) [upload=289762]: Got next segment. Inline: false Oct 7 06:47:01.782996983 (329h15m47.245089466s) [upload=289762]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:47:01.783001743 (329h15m47.245094226s) [upload=289762]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:47:02.026564536 (329h15m47.488657019s) [seg_pos={0 0}, upload=289762]: Begin upload segment... Oct 7 06:47:02.026640350 (329h15m47.488732833s) [seg_pos={0 0}, upload=289762]: Done begin upload segment. Oct 7 06:47:02.026687079 (329h15m47.488779583s) [noise=true , address=45.91.116.6:4973 , piece_id=KLJF4PKKG5QMMEIOUUUBU47YZGDI4CAJH3EPDMRECALJQGZUYEUQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.026701306 (329h15m47.488793789s) [noise=true , address=77.90.31.13:4627 , piece_id=KRA22MFRFMK3CTOM3MTHP3NXBKEFUNKILN53JS45NG4RLM3ZAYUQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.026767372 (329h15m47.488859865s) [noise=true , address=77.90.31.10:4520 , piece_id=P5AKDCDMRTAGIJDM6KZF5FZAL3QC2IQBOFZY3U5AIT346EC3CCSQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.027770452 (329h15m47.489862945s) [noise=true , address=45.91.116.11:4119 , piece_id=2WGKGT4Y6WKWF3OKESM2G6HEOAWBTCY4EETAUZQZIG2O3MJIDQGQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.028084370 (329h15m47.490176853s) [noise=true , address=45.91.116.14:4191 , piece_id=4ZAHXSCH26ZH7U5SD375WIC6LP6VX3PCISVB75KIBKCF3GIQKP3Q, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.028443463 (329h15m47.490535957s) [noise=true , address=145.79.227.18:4005 , piece_id=RI2HNTUY3JMZ2OKLR2XQHOU2IL2EPP6NAUF4QL3PS42AZKIHKWZA, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.028592687 (329h15m47.49068517s) [noise=true , address=77.90.31.18:4805 , piece_id=6SNS5JE7ECTX4OIVIY467GHOAHM6AJK64GOIFAXKPPYH25BKJ5DQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.029432827 (329h15m47.49152531s) [noise=true , address=145.79.227.3:4151 , piece_id=CAI2ZIEONAUPJG34DNY3VJM5L2YMTXWLK2ROISZD2GJJVCJISLYQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.030285541 (329h15m47.492378044s) [noise=true , address=145.79.227.12:4767 , piece_id=K26NI4OVGKZLMGF6D3W3QK4LNI3CAK2OK7C7YSEU33MEH3AEAQZQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.032516900 (329h15m47.494609383s) [noise=true , address=45.91.116.8:4047 , piece_id=2OOSQQL3Q2KHA7PP7DPXSXFMFKABNGV2KXSPSMQSIYTYWYUZR77A, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.033035638 (329h15m47.495128131s) [noise=true , address=145.79.227.14:4840 , piece_id=K4LKPHZDUIWIF2XPZYVMU3P23IH5MZR5MZGRZZG6BQATHAUN67SQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.033160035 (329h15m47.495252518s) [noise=true , address=77.90.31.15:4697 , piece_id=LGFHGZ5OGIOLJLOQ5LRSC34UHXVWSEV4CJGE6C6C3EBBEGRWDWUQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.033593961 (329h15m47.495686454s) [noise=true , address=45.91.116.4:4937 , piece_id=FYBJX5ZCIDVFRRIIX3MM3WHPKRDQ2PVMV4PWPEV6LM64KOBCFSBQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.034051793 (329h15m47.496144276s) [noise=true , address=45.91.116.13:4155 , piece_id=Y2HY7H54MADYAZMCYBS2LTYRBADGYQ7ERFKJXSCSGHVCGGXGKSZA, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.034554330 (329h15m47.496646813s) [noise=true , address=145.79.227.15:4873 , piece_id=BBTHUQLKGMSC4D5SLVW5JQ57VMBMMPGHWAQAJCOHB7IY2W6PM6WQ, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:02.034901882 (329h15m47.496994365s) [noise=true , address=77.90.31.21:4879 , piece_id=XJKMLZLOJ4OAEU6VYLA5IUA27JHMS4DTR6K4V7ZS3XZQOVKX3U7Q, seg_pos={0 0}, upload=289762]: Uploading piece... Oct 7 06:47:03.764896895 (329h15m49.226989408s) [upload=289762]: Next returned nil segment Oct 7 06:47:03.764923456 (329h15m49.227015939s) [upload=289762]: Waiting for error group managing segments... Oct 7 06:47:04.130941844 (329h15m49.593034327s) [noise=true , address=77.90.31.18:4805 , piece_id=6SNS5JE7ECTX4OIVIY467GHOAHM6AJK64GOIFAXKPPYH25BKJ5DQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.150561584 (329h15m49.612654077s) [noise=true , address=45.91.116.6:4973 , piece_id=KLJF4PKKG5QMMEIOUUUBU47YZGDI4CAJH3EPDMRECALJQGZUYEUQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.155672637 (329h15m49.61776512s) [noise=true , address=45.91.116.4:4937 , piece_id=FYBJX5ZCIDVFRRIIX3MM3WHPKRDQ2PVMV4PWPEV6LM64KOBCFSBQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.166510163 (329h15m49.628602646s) [noise=true , address=77.90.31.15:4697 , piece_id=LGFHGZ5OGIOLJLOQ5LRSC34UHXVWSEV4CJGE6C6C3EBBEGRWDWUQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.181528350 (329h15m49.643620833s) [noise=true , address=77.90.31.10:4520 , piece_id=P5AKDCDMRTAGIJDM6KZF5FZAL3QC2IQBOFZY3U5AIT346EC3CCSQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.196320486 (329h15m49.65841297s) [noise=true , address=77.90.31.13:4627 , piece_id=KRA22MFRFMK3CTOM3MTHP3NXBKEFUNKILN53JS45NG4RLM3ZAYUQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.254441709 (329h15m49.716534192s) [noise=true , address=45.91.116.13:4155 , piece_id=Y2HY7H54MADYAZMCYBS2LTYRBADGYQ7ERFKJXSCSGHVCGGXGKSZA, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.268867738 (329h15m49.730960231s) [noise=true , address=45.91.116.8:4047 , piece_id=2OOSQQL3Q2KHA7PP7DPXSXFMFKABNGV2KXSPSMQSIYTYWYUZR77A, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.293497878 (329h15m49.755590372s) [noise=true , address=45.91.116.11:4119 , piece_id=2WGKGT4Y6WKWF3OKESM2G6HEOAWBTCY4EETAUZQZIG2O3MJIDQGQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:04.901884125 (329h15m50.363976608s) [noise=true , address=77.90.31.21:4879 , piece_id=XJKMLZLOJ4OAEU6VYLA5IUA27JHMS4DTR6K4V7ZS3XZQOVKX3U7Q, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:05.158623489 (329h15m50.620715962s) [noise=true , address=45.91.116.14:4191 , piece_id=4ZAHXSCH26ZH7U5SD375WIC6LP6VX3PCISVB75KIBKCF3GIQKP3Q, seg_pos={0 0}, upload=289762]: Done uploading piece. err: <nil> Oct 7 06:47:31.079407372 (329h16m16.541499845s) [noise=true , address=145.79.227.18:4005 , piece_id=RI2HNTUY3JMZ2OKLR2XQHOU2IL2EPP6NAUF4QL3PS42AZKIHKWZA, seg_pos={0 0}, upload=289762]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: read tcp 188.166.225.170:38144->145.79.227.18:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:31.079511440 (329h16m16.541603913s) [noise=true , address=145.79.227.3:4151 , piece_id=CAI2ZIEONAUPJG34DNY3VJM5L2YMTXWLK2ROISZD2GJJVCJISLYQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: rpc: dial tcp 145.79.227.3:4151: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:31.079729044 (329h16m16.541821528s) [noise=true , address=145.79.227.15:4873 , piece_id=BBTHUQLKGMSC4D5SLVW5JQ57VMBMMPGHWAQAJCOHB7IY2W6PM6WQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: ecclient: upload failed (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3, address:145.79.227.15:4873): protocol: read tcp 188.166.225.170:57248->145.79.227.15:4873: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:31.079745585 (329h16m16.541838059s) [noise=true , address=145.79.227.12:4767 , piece_id=K26NI4OVGKZLMGF6D3W3QK4LNI3CAK2OK7C7YSEU33MEH3AEAQZQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:38590->145.79.227.12:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:31.079840557 (329h16m16.54193304s) [noise=true , address=145.79.227.14:4840 , piece_id=K4LKPHZDUIWIF2XPZYVMU3P23IH5MZR5MZGRZZG6BQATHAUN67SQ, seg_pos={0 0}, upload=289762]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: read tcp 188.166.225.170:53102->145.79.227.14:4840: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:47:31.079894901 (329h16m16.541987384s) [seg_pos={0 0}, upload=289762]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 06:47:31.317888088 (329h16m16.779980571s) [upload=289762]: Done uploading segments... Oct 7 06:47:31.317915500 (329h16m16.780007973s) [upload=289762]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.24065798s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................X....X.XX] duration: 1m0.245834306s FAIL (2025-10-07 06:46:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== ece... Oct 7 06:45:37.177895341 (329h14m22.639987824s) [noise=true , address=45.91.116.8:4078 , piece_id=IQY5WQUSXEOWTQIRQMYVJGXAIM6XQ5FHFLIXAI6GS6LNWT3QJNCQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.178319889 (329h14m22.640412372s) [noise=true , address=145.79.227.33:4869 , piece_id=JJEDNVJVY3CLB34KJDPGLNGBXTAB4CJDM42R5X57TQSUSHVQPDIQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.178392488 (329h14m22.640484971s) [noise=true , address=45.91.116.11:4143 , piece_id=NSY3BUM2WNP3XCWR52RUISSZO7IFIMUISR6KZIYXQPPTJQ2HAOYA, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.178715222 (329h14m22.640807705s) [noise=true , address=77.90.31.12:4613 , piece_id=JPEYSVN5FXPTZYSQSJN6D36YS3V352YMQFH3INDREOBM4S3VWZ5Q, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.178817637 (329h14m22.64091012s) [noise=true , address=145.79.227.27:4719 , piece_id=M47KBKMZMHYCZGKSPUGBUNRBUNVZZIJX6UNWYUC6ZD3F6GMCFOSQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.179115805 (329h14m22.641208278s) [noise=true , address=45.91.116.6:4981 , piece_id=DWFUTUN5IEIQFAAOZDDEI32LRQNXVNKKL4PNA5CVCGHUZMYC67LQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.179246984 (329h14m22.641339467s) [noise=true , address=145.79.227.35:4509 , piece_id=NCY7R3TRJUOJGJSV67US3LVR7HQFX2W7YAD2OVUKKGMIGGGH6TIQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.179473346 (329h14m22.641565829s) [noise=true , address=77.90.31.14:4665 , piece_id=CFVTCTPIVW3RYHYP5GGYILMCSQ2T3C2B5FKYRLYMXGBC4QN4UVNA, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.179738120 (329h14m22.641830603s) [noise=true , address=145.79.227.30:4782 , piece_id=ZTIPF74SBFYTCXUXUJC52HVUXFQUIA6A7NFKOHPB6Q2VOZFAB4VA, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.179860643 (329h14m22.641953126s) [noise=true , address=45.91.116.12:4234 , piece_id=2UL5XXTSB4YRXGUO3GD6KDRW5LB6EW3MT4A4O4U7AM4AFK5YXBQA, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.180222662 (329h14m22.642315155s) [noise=true , address=77.90.31.15:4714 , piece_id=LFQ24QOB4AVPCINEW4OJ3HOPKDWAS7CIZQJTA7GIXQY632RQXSAQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.180410711 (329h14m22.642503194s) [noise=true , address=145.79.227.36:4540 , piece_id=535K3V46P5PPTANVX6EEWXGPCKIJBIFCUVEP2KLK65LOFGJ5DLDQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.180581796 (329h14m22.64267428s) [noise=true , address=145.79.227.41:4096 , piece_id=AHIBJET5SXDF2GLYTGPWPMHKUVFDW3HWKXXWU7TALHGQZAGF2JJQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.181027314 (329h14m22.643119807s) [noise=true , address=145.79.227.25:4636 , piece_id=6NLFWAJQJPYKGMAILHMV3PNDAGAPMTLCPIU6M7QCTGVVF7TZWHJQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.181402258 (329h14m22.643494741s) [noise=true , address=45.91.116.13:4177 , piece_id=BG7TM6RGVABPNNDTSOMFJLYBK5VT74XNL26ULN7VXUWMM7ZHGBGQ, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:37.181770540 (329h14m22.643863033s) [noise=true , address=77.90.31.19:4859 , piece_id=ZRNLZVEGTVF5RSLN4UC6FRASFYFILGJ3DYG2CQVQLQE2BMQWQSMA, seg_pos={0 0}, upload=289742]: Uploading piece... Oct 7 06:45:38.711079464 (329h14m24.173171947s) [upload=289742]: Next returned nil segment Oct 7 06:45:38.711114641 (329h14m24.173207124s) [upload=289742]: Waiting for error group managing segments... Oct 7 06:45:39.102810884 (329h14m24.564903357s) [noise=true , address=45.91.116.6:4981 , piece_id=DWFUTUN5IEIQFAAOZDDEI32LRQNXVNKKL4PNA5CVCGHUZMYC67LQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.104867271 (329h14m24.566959754s) [noise=true , address=45.91.116.12:4234 , piece_id=2UL5XXTSB4YRXGUO3GD6KDRW5LB6EW3MT4A4O4U7AM4AFK5YXBQA, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.139141334 (329h14m24.601233807s) [noise=true , address=77.90.31.15:4714 , piece_id=LFQ24QOB4AVPCINEW4OJ3HOPKDWAS7CIZQJTA7GIXQY632RQXSAQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.145987951 (329h14m24.608080424s) [noise=true , address=45.91.116.11:4143 , piece_id=NSY3BUM2WNP3XCWR52RUISSZO7IFIMUISR6KZIYXQPPTJQ2HAOYA, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.150425130 (329h14m24.612517613s) [noise=true , address=77.90.31.12:4613 , piece_id=JPEYSVN5FXPTZYSQSJN6D36YS3V352YMQFH3INDREOBM4S3VWZ5Q, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.186473231 (329h14m24.648565704s) [noise=true , address=45.91.116.13:4177 , piece_id=BG7TM6RGVABPNNDTSOMFJLYBK5VT74XNL26ULN7VXUWMM7ZHGBGQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.222834380 (329h14m24.684926853s) [noise=true , address=77.90.31.11:4567 , piece_id=EJ2UTPTTZUJJXOVBN73LVVJWHB5XG6Q6YAQP4BZJBJLP2TFBA5JA, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.287676239 (329h14m24.749768712s) [noise=true , address=77.90.31.14:4665 , piece_id=CFVTCTPIVW3RYHYP5GGYILMCSQ2T3C2B5FKYRLYMXGBC4QN4UVNA, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:39.330101684 (329h14m24.792194167s) [noise=true , address=45.91.116.8:4078 , piece_id=IQY5WQUSXEOWTQIRQMYVJGXAIM6XQ5FHFLIXAI6GS6LNWT3QJNCQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:45:41.891366420 (329h14m27.353458903s) [noise=true , address=77.90.31.19:4859 , piece_id=ZRNLZVEGTVF5RSLN4UC6FRASFYFILGJ3DYG2CQVQLQE2BMQWQSMA, seg_pos={0 0}, upload=289742]: Done uploading piece. err: <nil> Oct 7 06:46:36.217712473 (329h15m21.679804946s) [noise=true , address=145.79.227.30:4782 , piece_id=ZTIPF74SBFYTCXUXUJC52HVUXFQUIA6A7NFKOHPB6Q2VOZFAB4VA, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:1vst2WW2D3NGbxPDDkjFQPxuwX3joAY77EdWCGXxU3H1NVGgcr, address:145.79.227.30:4782): protocol: read tcp 188.166.225.170:42534->145.79.227.30:4782: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.217854524 (329h15m21.679946997s) [noise=true , address=145.79.227.27:4719 , piece_id=M47KBKMZMHYCZGKSPUGBUNRBUNVZZIJX6UNWYUC6ZD3F6GMCFOSQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:12r5RXjrbpRcyxvcYhpoqQdgv8MD8zLmpninS7D3cdNCid4Rc8j, address:145.79.227.27:4719): protocol: read tcp 188.166.225.170:48424->145.79.227.27:4719: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.217907975 (329h15m21.680000448s) [noise=true , address=145.79.227.36:4540 , piece_id=535K3V46P5PPTANVX6EEWXGPCKIJBIFCUVEP2KLK65LOFGJ5DLDQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:1P92CRZqnx1avdDo18tMY2X7FujkZugb4gU98AaErD13gX4cLA, address:145.79.227.36:4540): protocol: read tcp 188.166.225.170:43262->145.79.227.36:4540: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.217986736 (329h15m21.680079209s) [noise=true , address=145.79.227.33:4869 , piece_id=JJEDNVJVY3CLB34KJDPGLNGBXTAB4CJDM42R5X57TQSUSHVQPDIQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:18CQxjq65r6xVTmNRurSQPzKxj4BoR6cJ8Uq29sTrdh6uXm5sq, address:145.79.227.33:4869): protocol: read tcp 188.166.225.170:47972->145.79.227.33:4869: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.218046459 (329h15m21.680138942s) [noise=true , address=145.79.227.24:4583 , piece_id=ZHZ2ZYNBGUBN4BM2C7TDHPQISJW7XPTMPCJH6F5APXY6FEDHOUOQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:1MCX7LMdSk3iENRGS5tE33ViZbANXGu3Po8bpenu4FLNGRQX19, address:145.79.227.24:4583): protocol: read tcp 188.166.225.170:41638->145.79.227.24:4583: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.218081606 (329h15m21.680174079s) [noise=true , address=145.79.227.41:4096 , piece_id=AHIBJET5SXDF2GLYTGPWPMHKUVFDW3HWKXXWU7TALHGQZAGF2JJQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:1izNXqYvnRLt96tcnR1TXt29qcxZWEgACmHuJ6CikSgLbAzxLL, address:145.79.227.41:4096): protocol: read tcp 188.166.225.170:39066->145.79.227.41:4096: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.218169313 (329h15m21.680261796s) [noise=true , address=145.79.227.25:4636 , piece_id=6NLFWAJQJPYKGMAILHMV3PNDAGAPMTLCPIU6M7QCTGVVF7TZWHJQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:1Lx4f6jtUdcZd9mVy5DgWqZ2vd4SrpSviaT7WUedxs7PLyzfHZ, address:145.79.227.25:4636): protocol: read tcp 188.166.225.170:59904->145.79.227.25:4636: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.218189240 (329h15m21.680281723s) [noise=true , address=145.79.227.35:4509 , piece_id=NCY7R3TRJUOJGJSV67US3LVR7HQFX2W7YAD2OVUKKGMIGGGH6TIQ, seg_pos={0 0}, upload=289742]: Done uploading piece. err: ecclient: upload failed (node:1y2NmauNvCyrMJ5z66Lihb3h3C9nwdJYqfSfNmC5dwDGYQqUKh, address:145.79.227.35:4509): protocol: read tcp 188.166.225.170:41798->145.79.227.35:4509: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:46:36.218229007 (329h15m21.68032149s) [seg_pos={0 0}, upload=289742]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Oct 7 06:46:36.461330762 (329h15m21.923423245s) [upload=289742]: Done uploading segments... Oct 7 06:46:36.461365328 (329h15m21.923457811s) [upload=289742]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.245849696s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [................................................X.....XXXXXX] duration: 30.002853603s FAIL (2025-10-07 06:46:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12W5Bso6XAuE6qzduTxCUmuzHWnUvNrjiwvogysvBLisdHEFjQQ, Piece ID: XIBORLXQVU7N5F7JS5IPSPQGS3EE2CDEAFANAC4XJP7CYPF6JX2Q) context deadline exceeded; piecestore: (Node ID: 1TP3JkhssLqPE6onMrKvTwA49PQUBJfuZWcPnGUeJd8UqR8zrF, Piece ID: RI6HGA2DRMUF5WL4OWHCOBZXO2QZNBSL2FVL6SX6XZTQEQXMKLTA) context deadline exceeded; piecestore: (Node ID: 1VJaB7i7HKyemCadiPSoXCuTuBzvqa5MeUDQf793etkYLpxQc9, Piece ID: FXZCWYIBBYKK6J4BLNWKSYLWBM7J5GGVUYTHQKQG2R3SXSOTQQXA) context deadline exceeded ======LOGS====== .435257066 (329h14m38.897349539s) [upload=289747]: Uploading segments... Oct 7 06:45:53.435319265 (329h14m38.897411738s) [upload=289747]: Got next segment. Inline: false Oct 7 06:45:53.435359762 (329h14m38.897452285s) [upload=289747]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:45:53.435368388 (329h14m38.897460871s) [upload=289747]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:45:53.696355580 (329h14m39.158448054s) [seg_pos={0 0}, upload=289747]: Begin upload segment... Oct 7 06:45:53.696434050 (329h14m39.158526543s) [seg_pos={0 0}, upload=289747]: Done begin upload segment. Oct 7 06:45:53.696457986 (329h14m39.158550459s) [noise=true , address=45.91.116.14:4191 , piece_id=OXRZYDA5A4ANAQ545PRSIHV37AOMB5PWBRAVWH746AI355UNOAAA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.696466231 (329h14m39.158558714s) [noise=true , address=77.90.31.18:4806 , piece_id=YRUBORQZB7TE3RP6JNIELB24QBCSGFT4DCTXOKDVRLGV5KBMA5SQ, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.696527037 (329h14m39.15861952s) [noise=true , address=45.91.116.7:4009 , piece_id=GCSBOUEPERFVHUNAK6MACUATQEMFGL3LZO2T43WVCZ5O2JZRKMYQ, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.697178378 (329h14m39.159270861s) [noise=true , address=77.90.31.21:4879 , piece_id=XIBORLXQVU7N5F7JS5IPSPQGS3EE2CDEAFANAC4XJP7CYPF6JX2Q, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.697568761 (329h14m39.159661244s) [noise=true , address=77.90.31.10:4517 , piece_id=BAPFIJM5ESYDKCRN6EABX75M43YVSHI7NECH3JMWUSSFDYOA26WQ, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.698142382 (329h14m39.160234876s) [noise=true , address=45.91.116.14:4189 , piece_id=BDVFBOO4TBLX7GM5YF2LCMN24Q7YSI2SGO3SA4MPS7JNWIXMFOWQ, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.698542325 (329h14m39.160634818s) [noise=true , address=45.91.116.6:4973 , piece_id=ZUDKMTZN2PPKXTNRVF3D7W4TEFZB3MOBCPG52XIG7I6AXF2WSMBA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.698953608 (329h14m39.161046091s) [noise=true , address=77.90.31.15:4697 , piece_id=VGB6LHZSRNU44LBHHO4A6ERJBB6XON6FKIWJFWMNADVBE4UN2KXA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.699443992 (329h14m39.161536475s) [noise=true , address=45.91.116.9:4263 , piece_id=RI6HGA2DRMUF5WL4OWHCOBZXO2QZNBSL2FVL6SX6XZTQEQXMKLTA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.700832575 (329h14m39.162925058s) [noise=true , address=45.91.116.8:4045 , piece_id=FXZCWYIBBYKK6J4BLNWKSYLWBM7J5GGVUYTHQKQG2R3SXSOTQQXA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.701188453 (329h14m39.163280926s) [noise=true , address=45.91.116.12:4227 , piece_id=M4HIT2FXYI5USWPY7QCOKIU6YSDDCEWD7BCPWXQB2BLM5MG4IIAQ, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.702414087 (329h14m39.16450657s) [noise=true , address=45.91.116.6:4974 , piece_id=XQBNAFCFCCBX46SHOG6VTPAMPGVTEQWKTSMYPWIKG4SBHCGCLVZA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.702810622 (329h14m39.164903105s) [noise=true , address=145.79.227.7:4293 , piece_id=LY7JVEHCDR6LLFPRNISUCLYSUWZT4K4K5BYWNJK7FX2PLMLAKOPA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.704201230 (329h14m39.166293713s) [noise=true , address=45.91.116.10:4082 , piece_id=FFDYHX6AK2YJ2WR45ZN7GSKQWY2RTQZMEH4LBKDQC7KDUV6ZZLAQ, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.704753010 (329h14m39.166845493s) [noise=true , address=45.91.116.12:4225 , piece_id=WTN2E3237I3FE4J2LVMZXDQJSWHI6PHAPHAJI3DMIEBRZICDDKZA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:53.706047205 (329h14m39.168139688s) [noise=true , address=145.79.227.19:4078 , piece_id=JHGQ2ZGQRFORYETRWZZ3IPZCBA55GJ32O2EZPPB3U3LAAFG7PBNA, seg_pos={0 0}, upload=289747]: Uploading piece... Oct 7 06:45:55.381019787 (329h14m40.84311227s) [upload=289747]: Next returned nil segment Oct 7 06:45:55.381046418 (329h14m40.843138941s) [upload=289747]: Waiting for error group managing segments... Oct 7 06:45:55.774309374 (329h14m41.236401857s) [noise=true , address=45.91.116.10:4082 , piece_id=FFDYHX6AK2YJ2WR45ZN7GSKQWY2RTQZMEH4LBKDQC7KDUV6ZZLAQ, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.779175901 (329h14m41.241268384s) [noise=true , address=45.91.116.6:4973 , piece_id=ZUDKMTZN2PPKXTNRVF3D7W4TEFZB3MOBCPG52XIG7I6AXF2WSMBA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.801231128 (329h14m41.263323611s) [noise=true , address=45.91.116.12:4225 , piece_id=WTN2E3237I3FE4J2LVMZXDQJSWHI6PHAPHAJI3DMIEBRZICDDKZA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.812533870 (329h14m41.274626353s) [noise=true , address=45.91.116.7:4009 , piece_id=GCSBOUEPERFVHUNAK6MACUATQEMFGL3LZO2T43WVCZ5O2JZRKMYQ, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.815297522 (329h14m41.277390015s) [noise=true , address=45.91.116.14:4189 , piece_id=BDVFBOO4TBLX7GM5YF2LCMN24Q7YSI2SGO3SA4MPS7JNWIXMFOWQ, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.818995034 (329h14m41.281087517s) [noise=true , address=77.90.31.18:4806 , piece_id=YRUBORQZB7TE3RP6JNIELB24QBCSGFT4DCTXOKDVRLGV5KBMA5SQ, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.852971630 (329h14m41.315064113s) [noise=true , address=45.91.116.12:4227 , piece_id=M4HIT2FXYI5USWPY7QCOKIU6YSDDCEWD7BCPWXQB2BLM5MG4IIAQ, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.891473134 (329h14m41.353565607s) [noise=true , address=45.91.116.6:4974 , piece_id=XQBNAFCFCCBX46SHOG6VTPAMPGVTEQWKTSMYPWIKG4SBHCGCLVZA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.893490996 (329h14m41.355583479s) [noise=true , address=77.90.31.15:4697 , piece_id=VGB6LHZSRNU44LBHHO4A6ERJBB6XON6FKIWJFWMNADVBE4UN2KXA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.904937222 (329h14m41.367029705s) [noise=true , address=45.91.116.14:4191 , piece_id=OXRZYDA5A4ANAQ545PRSIHV37AOMB5PWBRAVWH746AI355UNOAAA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.937631987 (329h14m41.39972446s) [noise=true , address=45.91.116.9:4263 , piece_id=RI6HGA2DRMUF5WL4OWHCOBZXO2QZNBSL2FVL6SX6XZTQEQXMKLTA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:55.942638791 (329h14m41.404731264s) [noise=true , address=77.90.31.21:4879 , piece_id=XIBORLXQVU7N5F7JS5IPSPQGS3EE2CDEAFANAC4XJP7CYPF6JX2Q, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:56.104140737 (329h14m41.56623322s) [noise=true , address=45.91.116.8:4045 , piece_id=FXZCWYIBBYKK6J4BLNWKSYLWBM7J5GGVUYTHQKQG2R3SXSOTQQXA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: <nil> Oct 7 06:45:56.104179341 (329h14m41.566271814s) [seg_pos={0 0}, upload=289747]: Segment reached optimal threshold of 13 pieces. Oct 7 06:45:56.104476968 (329h14m41.566569451s) [noise=true , address=77.90.31.10:4517 , piece_id=BAPFIJM5ESYDKCRN6EABX75M43YVSHI7NECH3JMWUSSFDYOA26WQ, seg_pos={0 0}, upload=289747]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:129FbXSqxqTJ99BdhfUePzbNvG8db4bxASiXQjSCbkErTaCEk7A): protocol: write tcp 188.166.225.170:35746->77.90.31.10:4517: use of closed network connection; write tcp 188.166.225.170:35746->77.90.31.10:4517: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35746->77.90.31.10:4517: use of closed network connection Oct 7 06:45:56.104502637 (329h14m41.56659511s) [noise=true , address=77.90.31.10:4517 , piece_id=BAPFIJM5ESYDKCRN6EABX75M43YVSHI7NECH3JMWUSSFDYOA26WQ, seg_pos={0 0}, upload=289747]: Long tail cancellation due to optimal threshold Oct 7 06:45:56.104577099 (329h14m41.566669592s) [noise=true , address=145.79.227.19:4078 , piece_id=JHGQ2ZGQRFORYETRWZZ3IPZCBA55GJ32O2EZPPB3U3LAAFG7PBNA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr): protocol: rpc: dial tcp 145.79.227.19:4078: operation was canceled; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:56.104588680 (329h14m41.566681153s) [noise=true , address=145.79.227.19:4078 , piece_id=JHGQ2ZGQRFORYETRWZZ3IPZCBA55GJ32O2EZPPB3U3LAAFG7PBNA, seg_pos={0 0}, upload=289747]: Long tail cancellation due to optimal threshold Oct 7 06:45:56.104629137 (329h14m41.56672161s) [noise=true , address=145.79.227.7:4293 , piece_id=LY7JVEHCDR6LLFPRNISUCLYSUWZT4K4K5BYWNJK7FX2PLMLAKOPA, seg_pos={0 0}, upload=289747]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: read tcp 188.166.225.170:45952->145.79.227.7:4293: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:56.104635470 (329h14m41.566727943s) [noise=true , address=145.79.227.7:4293 , piece_id=LY7JVEHCDR6LLFPRNISUCLYSUWZT4K4K5BYWNJK7FX2PLMLAKOPA, seg_pos={0 0}, upload=289747]: Long tail cancellation due to optimal threshold Oct 7 06:45:56.104680235 (329h14m41.566772718s) [seg_pos={0 0}, upload=289747]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:45:56.104795875 (329h14m41.566888348s) [upload=289747]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:45:56.104806254 (329h14m41.566898767s) [upload=289747]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:45:56.408461645 (329h14m41.870554128s) [upload=289747]: Done uploading segments... Oct 7 06:45:56.408488174 (329h14m41.870580657s) [upload=289747]: Upload finished. err: <nil> Duration: 30.002880124s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [.................................................X.....XXXXX] duration: 30.243429207s FAIL (2025-10-07 06:45:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== d Oct 7 06:45:11.439792953 (329h13m56.901885426s) [upload=289737]: Done starting upload Oct 7 06:45:11.439966313 (329h13m56.902058796s) [upload=289737]: Uploading segments... Oct 7 06:45:11.440111210 (329h13m56.902203693s) [upload=289737]: Got next segment. Inline: false Oct 7 06:45:11.440177115 (329h13m56.902269599s) [upload=289737]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:45:11.440182496 (329h13m56.902274979s) [upload=289737]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:45:11.690324907 (329h13m57.15241739s) [seg_pos={0 0}, upload=289737]: Begin upload segment... Oct 7 06:45:11.690421371 (329h13m57.152513854s) [seg_pos={0 0}, upload=289737]: Done begin upload segment. Oct 7 06:45:11.690455716 (329h13m57.152548199s) [noise=true , address=77.90.31.18:4805 , piece_id=NFKVVHG35WM2GPGJXJPKAIJPVXY53QQMX4PJTSJ3QAQ7FK6IOK7A, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.696171791 (329h13m57.158264274s) [noise=true , address=45.91.116.10:4083 , piece_id=MAV62VDTYBTWWHY5AYBRQLJETCZ4EIJJJ7GSPIYXZAW2UUW6OIQQ, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.698504883 (329h13m57.160597366s) [noise=true , address=45.91.116.9:4263 , piece_id=EFHJHDUJGKFU2Z3Q4R2COYC75ZYVAL7L7UMJJVL5SB25UJJNZTRQ, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.700145086 (329h13m57.162237569s) [noise=true , address=145.79.227.15:4874 , piece_id=3SZ3I6COUJ6DEKKLHZRZXTTJYKN5PLFKN2SBTKBKR4MYZC6UEVNA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.700860178 (329h13m57.162952661s) [noise=true , address=45.91.116.8:4048 , piece_id=DFIITAJWABD4X2BQI3577Y3CTU6YAI2H5X733NV2WQUQB3TCKNJA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.702398548 (329h13m57.164491031s) [noise=true , address=145.79.227.3:4150 , piece_id=MOROXURBLW6HXG5OYJPYGPQYTPQJ75EZEFTQWVJW6INBTWW4VXTA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.704160864 (329h13m57.166253347s) [noise=true , address=77.90.31.12:4589 , piece_id=EVPO6OYTHQWILMCMVNL25WTHZIPATU7S4D6AGXYMZVICYNS2NIMQ, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.704202252 (329h13m57.166294735s) [noise=true , address=45.91.116.12:4226 , piece_id=AHGYBXRJVCKMJP7RHTXKOXOTFYEPGZ2ANG73H47X2TKAQYMXHHVA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.705941194 (329h13m57.168033677s) [noise=true , address=145.79.227.13:4802 , piece_id=CKRMPPZIHNFJ4F6JEKFCCNIN2AC7IOVBZOSSVUKPNNTAKT6DRRLA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.706918795 (329h13m57.169011278s) [noise=true , address=145.79.227.5:4222 , piece_id=ZVURTQRJOPXTBCZBE6ZT2TH5HU5LU5S5EDWYNE42PBLQCXLEPC6Q, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.707678310 (329h13m57.169770824s) [noise=true , address=45.91.116.11:4117 , piece_id=ZABYJYOETNPDYJAJIVIYTZOG52SQJQ2NOZRMSAHMX7M2Y7P5POZQ, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.709438873 (329h13m57.171531346s) [noise=true , address=145.79.227.5:4221 , piece_id=ULVSGVQKH33N3K6NBB4QB4V2VJ7HCUMB35VBSS5TMN4VBMJPYUXQ, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.712853205 (329h13m57.174945688s) [noise=true , address=77.90.31.10:4519 , piece_id=QZZQ3XFKJSBUHL67XLMPMVF6WTNQP3PT4LJIE7I5QAOVHQZO4SQA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.714343813 (329h13m57.176436296s) [noise=true , address=145.79.227.4:4186 , piece_id=OMM6FLXLJGIUJ3I7WSHLT2BT5H2NSY644W5KRUCHRBHYK6YQLVSA, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.716133431 (329h13m57.178225914s) [noise=true , address=45.91.116.6:4974 , piece_id=PI4GKHJB7OC5XO36MTKVB26WEYGRUW4AMB7YG22PRXIRQNZI6D7A, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:11.718007028 (329h13m57.180099511s) [noise=true , address=145.79.227.2:4115 , piece_id=4G5B3ZOFBQN7552NPS2YPIXQUSFMA6AD46ACATR53W45SB6FBVYQ, seg_pos={0 0}, upload=289737]: Uploading piece... Oct 7 06:45:13.469940968 (329h13m58.932033451s) [upload=289737]: Next returned nil segment Oct 7 06:45:13.469996052 (329h13m58.932088575s) [upload=289737]: Waiting for error group managing segments... Oct 7 06:45:13.752185918 (329h13m59.214278391s) [noise=true , address=45.91.116.10:4083 , piece_id=MAV62VDTYBTWWHY5AYBRQLJETCZ4EIJJJ7GSPIYXZAW2UUW6OIQQ, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:13.774519976 (329h13m59.236612449s) [noise=true , address=45.91.116.8:4048 , piece_id=DFIITAJWABD4X2BQI3577Y3CTU6YAI2H5X733NV2WQUQB3TCKNJA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:13.799108235 (329h13m59.261200728s) [noise=true , address=45.91.116.11:4117 , piece_id=ZABYJYOETNPDYJAJIVIYTZOG52SQJQ2NOZRMSAHMX7M2Y7P5POZQ, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:13.873614088 (329h13m59.335706571s) [noise=true , address=45.91.116.6:4974 , piece_id=PI4GKHJB7OC5XO36MTKVB26WEYGRUW4AMB7YG22PRXIRQNZI6D7A, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:13.880471485 (329h13m59.342563969s) [noise=true , address=77.90.31.18:4805 , piece_id=NFKVVHG35WM2GPGJXJPKAIJPVXY53QQMX4PJTSJ3QAQ7FK6IOK7A, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:13.901600828 (329h13m59.363693332s) [noise=true , address=77.90.31.12:4589 , piece_id=EVPO6OYTHQWILMCMVNL25WTHZIPATU7S4D6AGXYMZVICYNS2NIMQ, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:14.055896083 (329h13m59.517988566s) [noise=true , address=77.90.31.10:4519 , piece_id=QZZQ3XFKJSBUHL67XLMPMVF6WTNQP3PT4LJIE7I5QAOVHQZO4SQA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:14.173174821 (329h13m59.635267314s) [noise=true , address=45.91.116.12:4226 , piece_id=AHGYBXRJVCKMJP7RHTXKOXOTFYEPGZ2ANG73H47X2TKAQYMXHHVA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:14.644498659 (329h14m0.106591142s) [noise=true , address=45.91.116.9:4263 , piece_id=EFHJHDUJGKFU2Z3Q4R2COYC75ZYVAL7L7UMJJVL5SB25UJJNZTRQ, seg_pos={0 0}, upload=289737]: Done uploading piece. err: <nil> Oct 7 06:45:40.745958847 (329h14m26.20805133s) [noise=true , address=145.79.227.4:4186 , piece_id=OMM6FLXLJGIUJ3I7WSHLT2BT5H2NSY644W5KRUCHRBHYK6YQLVSA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: read tcp 188.166.225.170:51268->145.79.227.4:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746134602 (329h14m26.208227085s) [noise=true , address=145.79.227.3:4150 , piece_id=MOROXURBLW6HXG5OYJPYGPQYTPQJ75EZEFTQWVJW6INBTWW4VXTA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: read tcp 188.166.225.170:51100->145.79.227.3:4150: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746286982 (329h14m26.208379455s) [noise=true , address=145.79.227.13:4802 , piece_id=CKRMPPZIHNFJ4F6JEKFCCNIN2AC7IOVBZOSSVUKPNNTAKT6DRRLA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: read tcp 188.166.225.170:56092->145.79.227.13:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746354791 (329h14m26.208447264s) [noise=true , address=145.79.227.5:4222 , piece_id=ZVURTQRJOPXTBCZBE6ZT2TH5HU5LU5S5EDWYNE42PBLQCXLEPC6Q, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: read tcp 188.166.225.170:49888->145.79.227.5:4222: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746407472 (329h14m26.208499955s) [noise=true , address=145.79.227.15:4874 , piece_id=3SZ3I6COUJ6DEKKLHZRZXTTJYKN5PLFKN2SBTKBKR4MYZC6UEVNA, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:50730->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746415887 (329h14m26.20850836s) [noise=true , address=145.79.227.2:4115 , piece_id=4G5B3ZOFBQN7552NPS2YPIXQUSFMA6AD46ACATR53W45SB6FBVYQ, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: read tcp 188.166.225.170:36808->145.79.227.2:4115: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746442829 (329h14m26.208535312s) [noise=true , address=145.79.227.5:4221 , piece_id=ULVSGVQKH33N3K6NBB4QB4V2VJ7HCUMB35VBSS5TMN4VBMJPYUXQ, seg_pos={0 0}, upload=289737]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:35584->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:40.746485149 (329h14m26.208577632s) [seg_pos={0 0}, upload=289737]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 06:45:40.987449134 (329h14m26.449541617s) [upload=289737]: Done uploading segments... Oct 7 06:45:40.987479652 (329h14m26.449572125s) [upload=289737]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.243469002s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X....X.X] duration: 1m0.249829342s FAIL (2025-10-07 06:45:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== oad segment... Oct 7 06:44:36.699215952 (329h13m22.161308435s) [seg_pos={0 0}, upload=289730]: Done begin upload segment. Oct 7 06:44:36.699246199 (329h13m22.161338682s) [noise=true , address=145.79.227.25:4625 , piece_id=LVXDRW3Q5KPAXW65TMIXV3PPE64JJMQVMJHCBKCTQJC2YDEBMTVQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.699277107 (329h13m22.161369591s) [noise=true , address=45.91.116.12:4230 , piece_id=FLJZU2TPXQ74GCJFT7CRIIAIHV6WXYLUVWFNWKEBF6SDNVUENU2Q, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.699361719 (329h13m22.161454192s) [noise=true , address=45.91.116.10:4113 , piece_id=3O47IHRJ4RJNNO5TFGR6SV6L3JSHCLRWATD24IC4CGIUN4WEDEHQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.699833066 (329h13m22.161925549s) [noise=true , address=77.90.31.16:4750 , piece_id=BEANSQWCYGW4FR5V5PHP7RX3CWL26PBYR3JUOQ6CC5KL24GDGPCA, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.700219923 (329h13m22.162312406s) [noise=true , address=45.91.116.5:4309 , piece_id=IDQIMDNSRGHXVMCODXMCB3OZRCGGN33H43SNWLE4SHDMFSQOEJJQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.700636386 (329h13m22.162728859s) [noise=true , address=64.49.13.155:4924 , piece_id=MPND2NUIVPQ5QCJUYEZBRM5LZJALAJM66JRN5IVTXQFRXSPOBN7Q, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.701076364 (329h13m22.163168847s) [noise=true , address=77.90.31.18:4829 , piece_id=HV43V42C4RSURV5KK3NNSQSUWVTIWCDUJAPZVYPNIELLYHTA2PIQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.701514138 (329h13m22.163606621s) [noise=true , address=145.79.227.16:4923 , piece_id=EVFSALMYWN7HAUFDLX754KNER4B6NB3I2T6CE5UE5G6QRD3XDGFQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.701915532 (329h13m22.164008015s) [noise=true , address=145.79.227.20:4449 , piece_id=OUYC7UW32UFS7375EJO672MQBSQOROL5C3HF6AGJBITWHOFCZB5Q, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.701945930 (329h13m22.164038423s) [noise=true , address=64.49.13.140:4505 , piece_id=FM4GMBMWHVTF3MS52OPISBUAV4QIBTPLQFTNGQDS4WM2YJQNODRQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.702330752 (329h13m22.164423235s) [noise=true , address=145.79.227.32:4865 , piece_id=W7DMZWBMEHBTN4KN4VQKUQVENAE3HD6AUENFPGJRRABWYFZAT4SQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.702806569 (329h13m22.164899052s) [noise=true , address=77.90.31.10:4543 , piece_id=62SZKFP4UGFMZEASFVP7OCRNT7GI2UTIXZWZS3AV4SBUL7EYOBRA, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.703544394 (329h13m22.165636877s) [noise=true , address=77.90.31.21:4900 , piece_id=HQZGDZY3NWV6OBPL7CLF6WOXWI2MHPVR5HBXSTX56GIRWXEWX73A, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.704381297 (329h13m22.16647377s) [noise=true , address=45.91.116.8:4063 , piece_id=M5CAJGNHVMHX2YSPQHQKRRQUO4NDSN46QKQGVYUN3NKIHUHTUUFQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.704954578 (329h13m22.167047061s) [noise=true , address=45.91.116.13:4185 , piece_id=X4FHPLTKILMXNFFUWRCLXGFDGPOGHGIHKJWOY26IK7OAPBSSDPYQ, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.705385670 (329h13m22.167478153s) [noise=true , address=77.90.31.17:4780 , piece_id=SIHXRDOCICQXKWYVWT64B7REJQ5VXOQQVM3F57Z7IIBN6JWJ33FA, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.705880151 (329h13m22.167972634s) [noise=true , address=145.79.227.23:4566 , piece_id=5ONHBCBLYPNCO4Q62OYQE22KQ4O3X3ZDM4HMQUZWNZTALKSMW4GA, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:36.706383509 (329h13m22.168476002s) [noise=true , address=45.91.116.9:4294 , piece_id=6SFRSJ757XME4LUCOADJY5KPVINQ6BBUGTWFEIUEJXZBBCZU7KTA, seg_pos={0 0}, upload=289730]: Uploading piece... Oct 7 06:44:38.221245134 (329h13m23.683337607s) [upload=289730]: Next returned nil segment Oct 7 06:44:38.221277446 (329h13m23.683369929s) [upload=289730]: Waiting for error group managing segments... Oct 7 06:44:38.603073417 (329h13m24.065165901s) [noise=true , address=45.91.116.9:4294 , piece_id=6SFRSJ757XME4LUCOADJY5KPVINQ6BBUGTWFEIUEJXZBBCZU7KTA, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.675233923 (329h13m24.137326396s) [noise=true , address=45.91.116.8:4063 , piece_id=M5CAJGNHVMHX2YSPQHQKRRQUO4NDSN46QKQGVYUN3NKIHUHTUUFQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.681172291 (329h13m24.143264774s) [noise=true , address=77.90.31.10:4543 , piece_id=62SZKFP4UGFMZEASFVP7OCRNT7GI2UTIXZWZS3AV4SBUL7EYOBRA, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.693189473 (329h13m24.155281956s) [noise=true , address=45.91.116.10:4113 , piece_id=3O47IHRJ4RJNNO5TFGR6SV6L3JSHCLRWATD24IC4CGIUN4WEDEHQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.703241033 (329h13m24.165333506s) [noise=true , address=45.91.116.5:4309 , piece_id=IDQIMDNSRGHXVMCODXMCB3OZRCGGN33H43SNWLE4SHDMFSQOEJJQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.716547070 (329h13m24.178639543s) [noise=true , address=77.90.31.16:4750 , piece_id=BEANSQWCYGW4FR5V5PHP7RX3CWL26PBYR3JUOQ6CC5KL24GDGPCA, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.749567106 (329h13m24.211659589s) [noise=true , address=77.90.31.21:4900 , piece_id=HQZGDZY3NWV6OBPL7CLF6WOXWI2MHPVR5HBXSTX56GIRWXEWX73A, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.778763170 (329h13m24.240855653s) [noise=true , address=77.90.31.17:4780 , piece_id=SIHXRDOCICQXKWYVWT64B7REJQ5VXOQQVM3F57Z7IIBN6JWJ33FA, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.900636747 (329h13m24.36272923s) [noise=true , address=45.91.116.12:4230 , piece_id=FLJZU2TPXQ74GCJFT7CRIIAIHV6WXYLUVWFNWKEBF6SDNVUENU2Q, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:38.919935375 (329h13m24.382027858s) [noise=true , address=77.90.31.18:4829 , piece_id=HV43V42C4RSURV5KK3NNSQSUWVTIWCDUJAPZVYPNIELLYHTA2PIQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:39.397366912 (329h13m24.859459395s) [noise=true , address=45.91.116.13:4185 , piece_id=X4FHPLTKILMXNFFUWRCLXGFDGPOGHGIHKJWOY26IK7OAPBSSDPYQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:44:52.077699903 (329h13m37.539792376s) [noise=true , address=64.49.13.155:4924 , piece_id=MPND2NUIVPQ5QCJUYEZBRM5LZJALAJM66JRN5IVTXQFRXSPOBN7Q, seg_pos={0 0}, upload=289730]: Done uploading piece. err: <nil> Oct 7 06:45:35.755533562 (329h14m21.217626055s) [noise=true , address=64.49.13.140:4505 , piece_id=FM4GMBMWHVTF3MS52OPISBUAV4QIBTPLQFTNGQDS4WM2YJQNODRQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: ecclient: upload failed (node:1AbFiw33iRaTCNbSt9NmCYG1KBqys85G5VcgXndydkzgnxN4L4, address:64.49.13.140:4505): protocol: read tcp 188.166.225.170:47534->64.49.13.140:4505: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:35.755651667 (329h14m21.21774415s) [noise=true , address=145.79.227.32:4865 , piece_id=W7DMZWBMEHBTN4KN4VQKUQVENAE3HD6AUENFPGJRRABWYFZAT4SQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: ecclient: upload failed (node:12DQ2GHBBNrbKsE6eo3YTsMkQGuekVYwrQQjBdQWtQWNgd35VPQ, address:145.79.227.32:4865): protocol: read tcp 188.166.225.170:36562->145.79.227.32:4865: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:35.755678177 (329h14m21.21777066s) [noise=true , address=145.79.227.25:4625 , piece_id=LVXDRW3Q5KPAXW65TMIXV3PPE64JJMQVMJHCBKCTQJC2YDEBMTVQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: ecclient: upload failed (node:1G6VA95626cVYC8TDGiqRQAfXYhywwcSy6YieMQoB6tqEvjSrF, address:145.79.227.25:4625): protocol: read tcp 188.166.225.170:35864->145.79.227.25:4625: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:35.755728563 (329h14m21.217821036s) [noise=true , address=145.79.227.23:4566 , piece_id=5ONHBCBLYPNCO4Q62OYQE22KQ4O3X3ZDM4HMQUZWNZTALKSMW4GA, seg_pos={0 0}, upload=289730]: Done uploading piece. err: ecclient: upload failed (node:1Z9ic1vvfDViKVZbrkLZnryBxKKQ4rJdjBFY6wEYCDtJt9ci77, address:145.79.227.23:4566): protocol: read tcp 188.166.225.170:60160->145.79.227.23:4566: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:35.755839875 (329h14m21.217932348s) [noise=true , address=145.79.227.16:4923 , piece_id=EVFSALMYWN7HAUFDLX754KNER4B6NB3I2T6CE5UE5G6QRD3XDGFQ, seg_pos={0 0}, upload=289730]: Done uploading piece. err: ecclient: upload failed (node:12Ee9ZfH81oy4SAUxozjQNWJ7ALmWDXQZYPLFwuAiJ8V6E7jT6r, address:145.79.227.16:4923): protocol: read tcp 188.166.225.170:52700->145.79.227.16:4923: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:35.755879641 (329h14m21.217972124s) [noise=true , address=145.79.227.20:4449 , piece_id=OUYC7UW32UFS7375EJO672MQBSQOROL5C3HF6AGJBITWHOFCZB5Q, seg_pos={0 0}, upload=289730]: Done uploading piece. err: ecclient: upload failed (node:1r53UYADNA4XuUzpWdAvUh6T3iUJSbX9YvqnimzoPTgMfSmBWF, address:145.79.227.20:4449): protocol: read tcp 188.166.225.170:51172->145.79.227.20:4449: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:45:35.755921300 (329h14m21.218013783s) [seg_pos={0 0}, upload=289730]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Oct 7 06:45:36.002537066 (329h14m21.464629569s) [upload=289730]: Done uploading segments... Oct 7 06:45:36.002579908 (329h14m21.464672401s) [upload=289730]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.249851204s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [..................................................X.....XXXX] duration: 30.24561802s FAIL (2025-10-07 06:44:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 06:44:28.993278554 (329h13m14.455371037s) [upload=289727]: Starting upload Oct 7 06:44:28.993358997 (329h13m14.45545148s) [upload=289727]: Done starting upload Oct 7 06:44:28.993584757 (329h13m14.455677241s) [upload=289727]: Uploading segments... Oct 7 06:44:28.993632288 (329h13m14.455724761s) [upload=289727]: Got next segment. Inline: false Oct 7 06:44:28.993679748 (329h13m14.455772231s) [upload=289727]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:44:28.993685508 (329h13m14.455777992s) [upload=289727]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:44:29.244510360 (329h13m14.706602843s) [seg_pos={0 0}, upload=289727]: Begin upload segment... Oct 7 06:44:29.244580123 (329h13m14.706672606s) [seg_pos={0 0}, upload=289727]: Done begin upload segment. Oct 7 06:44:29.244612964 (329h13m14.706705447s) [noise=true , address=145.79.227.18:4005 , piece_id=HLLQJGFBZD7DJWXUQU364H6JRMCDBZHPLPQPSIC5ZMUTJQCJGLMA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.244627062 (329h13m14.706719545s) [noise=true , address=45.91.116.9:4262 , piece_id=IADS6IMP6NCCUIX7GZUU53YLI6SI5L47RYVRWFWS3RTHQI7RRPZA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.244813797 (329h13m14.70690627s) [noise=true , address=77.90.31.16:4733 , piece_id=YA3ZRBRC3KMP7LDV5LNBRWNYOCXJRLNMYCDIIB5ZXNLTCMHIWAGA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.245563014 (329h13m14.707655497s) [noise=true , address=77.90.31.21:4877 , piece_id=37Y5TSDDDQUPXFWT3DRWNENPFTQWUCJEWMV7W7X74AJBNTZ4SB3Q, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.246011839 (329h13m14.708104312s) [noise=true , address=77.90.31.11:4553 , piece_id=QQBULUFTVH6YOJQ6OFPHDHT2W2WG6INLVF5DMOVXIPKRDRBCT2PA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.246412161 (329h13m14.708504644s) [noise=true , address=145.79.227.13:4804 , piece_id=KVUPAUZAEY3DWBSNH5P3KDDX6RI6DHLWIMIUHOTIW42H4COEOUDA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.246822232 (329h13m14.708914715s) [noise=true , address=77.90.31.15:4699 , piece_id=BRPVJH4RYQ3TESXC3ER7UPJXO43COEJHSWT67J3WHH7OKBQSXEAA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.247269443 (329h13m14.709361926s) [noise=true , address=45.91.116.4:4939 , piece_id=MPEWPHVNEOXXPY23YDSODZ54LUD2UO36TV5EB67RAXETG5KRNXGQ, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.247749117 (329h13m14.7098416s) [noise=true , address=145.79.227.16:4909 , piece_id=7HRJWUWSFVHVGZANMVW5TMHO5ZTJVHE23TTVWKX46PKVFTTLEYGA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.249171665 (329h13m14.711264148s) [noise=true , address=45.91.116.5:4298 , piece_id=BM5VDGOZI7JUKQCBOZEOGE3EW63YIENSXZNJLI4VV2HWF5FXQ7IA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.249419277 (329h13m14.71151176s) [noise=true , address=77.90.31.10:4519 , piece_id=3ZVTV2TD6YLERHUZM5F6N6A2VUWGHK3CO6ZLWTH3STZBXPJMVMOA, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.251006249 (329h13m14.713098722s) [noise=true , address=45.91.116.6:4975 , piece_id=D52L7O2OGQ2LQM3YW7YF3AGJLWFH3LSVB3Q7ALXG4ZMMIXO5S57A, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.251627271 (329h13m14.713719754s) [noise=true , address=45.91.116.7:4009 , piece_id=YHKBHSBM4BG2EJLLS4RDRIUGTHMREGAYLPEL2FKK6CPW3XWFHTRQ, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.252604332 (329h13m14.714696815s) [noise=true , address=145.79.227.12:4767 , piece_id=NVIR4RMZHCCNJCDGGBYPKZ2VHGWHZAM4QFAHSFRMQ4YC3UBACK7A, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.254006962 (329h13m14.716099445s) [noise=true , address=45.91.116.6:4974 , piece_id=U2U6A2GCHEWEDA7C5OMOQUSMBQWD5OCXQDIIFCML4DKTIELEKW6Q, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:29.255781361 (329h13m14.717873844s) [noise=true , address=45.91.116.13:4155 , piece_id=YHGLQTKG6WADDLEVC57XI7PHNV73SH7XK2FKXWOA6IA6GT3NJSSQ, seg_pos={0 0}, upload=289727]: Uploading piece... Oct 7 06:44:30.916886970 (329h13m16.378979443s) [upload=289727]: Next returned nil segment Oct 7 06:44:30.916904823 (329h13m16.378997296s) [upload=289727]: Waiting for error group managing segments... Oct 7 06:44:31.308280507 (329h13m16.77037299s) [noise=true , address=45.91.116.4:4939 , piece_id=MPEWPHVNEOXXPY23YDSODZ54LUD2UO36TV5EB67RAXETG5KRNXGQ, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.309867899 (329h13m16.771960392s) [noise=true , address=45.91.116.6:4974 , piece_id=U2U6A2GCHEWEDA7C5OMOQUSMBQWD5OCXQDIIFCML4DKTIELEKW6Q, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.311545214 (329h13m16.773637697s) [noise=true , address=77.90.31.16:4733 , piece_id=YA3ZRBRC3KMP7LDV5LNBRWNYOCXJRLNMYCDIIB5ZXNLTCMHIWAGA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.349551624 (329h13m16.811644097s) [noise=true , address=77.90.31.15:4699 , piece_id=BRPVJH4RYQ3TESXC3ER7UPJXO43COEJHSWT67J3WHH7OKBQSXEAA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.371207550 (329h13m16.833300033s) [noise=true , address=77.90.31.10:4519 , piece_id=3ZVTV2TD6YLERHUZM5F6N6A2VUWGHK3CO6ZLWTH3STZBXPJMVMOA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.387775068 (329h13m16.849867551s) [noise=true , address=45.91.116.9:4262 , piece_id=IADS6IMP6NCCUIX7GZUU53YLI6SI5L47RYVRWFWS3RTHQI7RRPZA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.394266690 (329h13m16.856359163s) [noise=true , address=45.91.116.6:4975 , piece_id=D52L7O2OGQ2LQM3YW7YF3AGJLWFH3LSVB3Q7ALXG4ZMMIXO5S57A, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.421769769 (329h13m16.883862262s) [noise=true , address=45.91.116.13:4155 , piece_id=YHGLQTKG6WADDLEVC57XI7PHNV73SH7XK2FKXWOA6IA6GT3NJSSQ, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.426105175 (329h13m16.888197658s) [noise=true , address=77.90.31.11:4553 , piece_id=QQBULUFTVH6YOJQ6OFPHDHT2W2WG6INLVF5DMOVXIPKRDRBCT2PA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.432227343 (329h13m16.894319826s) [noise=true , address=45.91.116.7:4009 , piece_id=YHKBHSBM4BG2EJLLS4RDRIUGTHMREGAYLPEL2FKK6CPW3XWFHTRQ, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.472248149 (329h13m16.934340632s) [noise=true , address=77.90.31.21:4877 , piece_id=37Y5TSDDDQUPXFWT3DRWNENPFTQWUCJEWMV7W7X74AJBNTZ4SB3Q, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:31.701737503 (329h13m17.163829996s) [noise=true , address=45.91.116.5:4298 , piece_id=BM5VDGOZI7JUKQCBOZEOGE3EW63YIENSXZNJLI4VV2HWF5FXQ7IA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: <nil> Oct 7 06:44:58.290991286 (329h13m43.753083789s) [noise=true , address=145.79.227.16:4909 , piece_id=7HRJWUWSFVHVGZANMVW5TMHO5ZTJVHE23TTVWKX46PKVFTTLEYGA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: read tcp 188.166.225.170:56670->145.79.227.16:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:44:58.291138837 (329h13m43.75323132s) [noise=true , address=145.79.227.12:4767 , piece_id=NVIR4RMZHCCNJCDGGBYPKZ2VHGWHZAM4QFAHSFRMQ4YC3UBACK7A, seg_pos={0 0}, upload=289727]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:51346->145.79.227.12:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:44:58.291335441 (329h13m43.753427924s) [noise=true , address=145.79.227.18:4005 , piece_id=HLLQJGFBZD7DJWXUQU364H6JRMCDBZHPLPQPSIC5ZMUTJQCJGLMA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: read tcp 188.166.225.170:55188->145.79.227.18:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:44:58.291479375 (329h13m43.753571858s) [noise=true , address=145.79.227.13:4804 , piece_id=KVUPAUZAEY3DWBSNH5P3KDDX6RI6DHLWIMIUHOTIW42H4COEOUDA, seg_pos={0 0}, upload=289727]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:42580->145.79.227.13:4804: use of closed network connection; context deadline exceeded Oct 7 06:44:58.291508651 (329h13m43.753601124s) [seg_pos={0 0}, upload=289727]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 06:44:58.533990646 (329h13m43.996083119s) [upload=289727]: Done uploading segments... Oct 7 06:44:58.534022226 (329h13m43.996114709s) [upload=289727]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.245648898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X....X.] duration: 37.643980852s FAIL (2025-10-07 06:44:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== NJI3KJCCE3V6GXTNDHQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.397792889 (329h11m43.859885373s) [noise=true , address=64.49.13.137:4501 , piece_id=GIQQRHGX63IPQTWVXNXFA6K5Y7CWRB2KKMZ4M7PLX5KF2746CYTQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.397968223 (329h11m43.860060706s) [noise=true , address=77.90.31.17:4779 , piece_id=5LAWE5T66ZE3JY5CUHUOX5SGAI6KKNIUTLAQHDJCGBHXVVP26OGA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.398390818 (329h11m43.860483301s) [noise=true , address=77.90.31.10:4537 , piece_id=B3ZCOJTSIBYJNQHWSPM5DJOW7GDI6EV3UGBGB5ZSOOKSVNQI3Y7A, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.398515486 (329h11m43.860607969s) [noise=true , address=145.79.227.52:4445 , piece_id=DLH5MKIQFGTU22BYLNZ6FWD43IKU4OPKWV5EZ6CXJIPRA3MKOBRA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.398799106 (329h11m43.860891599s) [noise=true , address=145.79.227.35:4489 , piece_id=4PDSJV66E5KZYMZJH2ZXTBGHLSC7TA67OOTR5HKMMMDNFXXT6PNQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399021328 (329h11m43.861113801s) [noise=true , address=145.79.227.26:4662 , piece_id=A3SIDE35VMW3HOKLID5VJ2PHRRFKX7AIFYOBU3ZTRKBZE2YNRTDQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399207743 (329h11m43.861300226s) [noise=true , address=145.79.227.22:4529 , piece_id=QGHGSDVHQTVF5UGHPWRPPKO4ZLBVYBOCQ3G7MOTEQTA4EZVO2S2A, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399525268 (329h11m43.861617752s) [noise=true , address=145.79.227.44:4201 , piece_id=PSMY6DXTGJ5J6H7O6Y2EKGCESB6D6BDXQTO3US7LIHMZM4TKIE5A, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399646208 (329h11m43.861738691s) [noise=true , address=77.90.31.19:4866 , piece_id=M3P6WWAVY2PO7F4GBASZVR5ET33DI5RQHFC3BUKQF6SVTZ3NOPLA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.400099010 (329h11m43.862191484s) [noise=true , address=77.90.31.15:4714 , piece_id=GDH2YXYYKUJAY2B7CKVFXDEB4MFLZLPCUXLL2O2DUKY5B45SQ3FA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.400147793 (329h11m43.862240276s) [noise=true , address=45.91.116.8:4069 , piece_id=INAZ2ZSSPJ5XKSY44IBMW36KHHQNKHTOUS5BO4SG5LURMVGWIZ7Q, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.401488466 (329h11m43.863580939s) [noise=true , address=77.90.31.18:4840 , piece_id=W4DPPNJQVCGBM7GHBKZKKR7DXNPQA4AW5ZIYOBD5S44PDHF3FPTA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.402290413 (329h11m43.864382896s) [noise=true , address=45.91.116.11:4124 , piece_id=UPOBMCYB6VMPSGQA2XOGPH5E3ACOG36I5SJ22C25TY4RQKOMWERA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.403080789 (329h11m43.865173272s) [noise=true , address=77.90.31.17:4780 , piece_id=7KNWVCIF45CTXKZ4ND52KNUOHD6S4RZ3OOAXJPVZY3IDRAUEQ7KA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.404309278 (329h11m43.866401761s) [noise=true , address=145.79.227.20:4454 , piece_id=QC7DHFH27O5Y72LCLPH5PG3VWWDY3A2FMO3W6YKI2VHA74BPIUDQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.405692161 (329h11m43.867784654s) [noise=true , address=77.90.31.13:4641 , piece_id=PB2JQE5QB7DYGVMEZRMF4PBV6FHMTSDPDE6525H54WOJZMPYATIQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:59.949497860 (329h11m45.411590343s) [upload=289707]: Next returned nil segment Oct 7 06:42:59.949522658 (329h11m45.411615151s) [upload=289707]: Waiting for error group managing segments... Oct 7 06:43:00.339907244 (329h11m45.801999727s) [noise=true , address=77.90.31.17:4780 , piece_id=7KNWVCIF45CTXKZ4ND52KNUOHD6S4RZ3OOAXJPVZY3IDRAUEQ7KA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.366592848 (329h11m45.828685331s) [noise=true , address=77.90.31.13:4641 , piece_id=PB2JQE5QB7DYGVMEZRMF4PBV6FHMTSDPDE6525H54WOJZMPYATIQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.373948594 (329h11m45.836041077s) [noise=true , address=45.91.116.8:4069 , piece_id=INAZ2ZSSPJ5XKSY44IBMW36KHHQNKHTOUS5BO4SG5LURMVGWIZ7Q, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.382656756 (329h11m45.844749239s) [noise=true , address=77.90.31.10:4537 , piece_id=B3ZCOJTSIBYJNQHWSPM5DJOW7GDI6EV3UGBGB5ZSOOKSVNQI3Y7A, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.394420667 (329h11m45.85651315s) [noise=true , address=77.90.31.18:4840 , piece_id=W4DPPNJQVCGBM7GHBKZKKR7DXNPQA4AW5ZIYOBD5S44PDHF3FPTA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.402582077 (329h11m45.86467456s) [noise=true , address=77.90.31.15:4714 , piece_id=GDH2YXYYKUJAY2B7CKVFXDEB4MFLZLPCUXLL2O2DUKY5B45SQ3FA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.430546977 (329h11m45.89263946s) [noise=true , address=77.90.31.19:4866 , piece_id=M3P6WWAVY2PO7F4GBASZVR5ET33DI5RQHFC3BUKQF6SVTZ3NOPLA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.433192705 (329h11m45.895285188s) [noise=true , address=77.90.31.14:4689 , piece_id=4H2XAFHUCU4V4BIYU3UKFVG54R736QH3PS4BA727EY76VWA6JK2Q, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.493973121 (329h11m45.956065604s) [noise=true , address=45.91.116.11:4124 , piece_id=UPOBMCYB6VMPSGQA2XOGPH5E3ACOG36I5SJ22C25TY4RQKOMWERA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.559588964 (329h11m46.021681447s) [noise=true , address=77.90.31.17:4779 , piece_id=5LAWE5T66ZE3JY5CUHUOX5SGAI6KKNIUTLAQHDJCGBHXVVP26OGA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:57.423779761 (329h12m42.885872244s) [noise=true , address=145.79.227.44:4201 , piece_id=PSMY6DXTGJ5J6H7O6Y2EKGCESB6D6BDXQTO3US7LIHMZM4TKIE5A, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi, address:145.79.227.44:4201): protocol: rpc: dial tcp 145.79.227.44:4201: i/o timeout; context deadline exceeded Oct 7 06:43:57.423882767 (329h12m42.88597524s) [noise=true , address=145.79.227.22:4529 , piece_id=QGHGSDVHQTVF5UGHPWRPPKO4ZLBVYBOCQ3G7MOTEQTA4EZVO2S2A, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12bNgTA4CY9ShB6Mi3QpceTEqNCr8ohCMHNVvU6S1qJp2TdVP1i, address:145.79.227.22:4529): protocol: read tcp 188.166.225.170:55814->145.79.227.22:4529: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.423959603 (329h12m42.886052076s) [noise=true , address=64.49.13.140:4513 , piece_id=LYF2S6TEPKQ3M6HGCKGCPCXIWVQOYZU56NJI3KJCCE3V6GXTNDHQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY, address:64.49.13.140:4513): protocol: write tcp 188.166.225.170:53248->64.49.13.140:4513: use of closed network connection; write tcp 188.166.225.170:53248->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->64.49.13.140:4513: use of closed network connection Oct 7 06:43:57.424030448 (329h12m42.886122921s) [noise=true , address=145.79.227.52:4445 , piece_id=DLH5MKIQFGTU22BYLNZ6FWD43IKU4OPKWV5EZ6CXJIPRA3MKOBRA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:43:57.424049665 (329h12m42.886142148s) [noise=true , address=64.49.13.137:4501 , piece_id=GIQQRHGX63IPQTWVXNXFA6K5Y7CWRB2KKMZ4M7PLX5KF2746CYTQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12D2MWhZKXEMUPBvMQVfzVJZNw4m857MYZ4BjLfEwVxS4fJLzj1, address:64.49.13.137:4501): protocol: read tcp 188.166.225.170:42886->64.49.13.137:4501: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.424269313 (329h12m42.886361796s) [noise=true , address=145.79.227.26:4662 , piece_id=A3SIDE35VMW3HOKLID5VJ2PHRRFKX7AIFYOBU3ZTRKBZE2YNRTDQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:1uyMAEh41FVYSVGkNjzweDWqHKaFzY9YvxKcCFJTJ4Y5cehKZV, address:145.79.227.26:4662): protocol: read tcp 188.166.225.170:60434->145.79.227.26:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.424306664 (329h12m42.886399147s) [noise=true , address=145.79.227.35:4489 , piece_id=4PDSJV66E5KZYMZJH2ZXTBGHLSC7TA67OOTR5HKMMMDNFXXT6PNQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12YbmXQeh182PYH1AHxvGcw3zW3uiuDsbYPomH3HAPNtjbJSpPw, address:145.79.227.35:4489): protocol: read tcp 188.166.225.170:32942->145.79.227.35:4489: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.424372870 (329h12m42.886465353s) [noise=true , address=145.79.227.20:4454 , piece_id=QC7DHFH27O5Y72LCLPH5PG3VWWDY3A2FMO3W6YKI2VHA74BPIUDQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J, address:145.79.227.20:4454): protocol: context deadline exceeded; context deadline exceeded Oct 7 06:43:57.424446831 (329h12m42.886539314s) [seg_pos={0 0}, upload=289707]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Oct 7 06:43:57.682720297 (329h12m43.14481279s) [upload=289707]: Done uploading segments... Oct 7 06:43:57.682759371 (329h12m43.144851864s) [upload=289707]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 37.64400032s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................X.....XXX] duration: 30.24629563s FAIL (2025-10-07 06:44:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 06:43:58.527459968 (329h12m43.989552441s) [upload=289719]: Starting upload Oct 7 06:43:58.527551262 (329h12m43.989643735s) [upload=289719]: Done starting upload Oct 7 06:43:58.527739260 (329h12m43.989831743s) [upload=289719]: Uploading segments... Oct 7 06:43:58.527778224 (329h12m43.989870697s) [upload=289719]: Got next segment. Inline: false Oct 7 06:43:58.527806177 (329h12m43.98989866s) [upload=289719]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:43:58.527810846 (329h12m43.989903329s) [upload=289719]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:43:58.776504199 (329h12m44.238596682s) [seg_pos={0 0}, upload=289719]: Begin upload segment... Oct 7 06:43:58.776577138 (329h12m44.238669621s) [seg_pos={0 0}, upload=289719]: Done begin upload segment. Oct 7 06:43:58.776612966 (329h12m44.238705449s) [noise=true , address=145.79.227.6:4258 , piece_id=IYRTJTFGWJD34TCTCK376NGUFJ2MLATVTB4HPSBX3R4JKSEP7YHA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.776776347 (329h12m44.238868831s) [noise=true , address=145.79.227.4:4187 , piece_id=4PTDZ5SDFBUUNG4JZOJTUL4JC3CWYKAGAWDAC3S67JDNRMAHT3CQ, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.777382762 (329h12m44.239475235s) [noise=true , address=77.90.31.10:4520 , piece_id=6K4DR5SSMJ74KZNLLTRLK2BM7HOMMO2RAOV7W5KC6F3UIAVAZ77A, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.777639080 (329h12m44.239731554s) [noise=true , address=45.91.116.14:4191 , piece_id=TTM7JBIKPREG7YULIFAOCSC5BRS46IC2TSVL6UBRYMN4I7TSRT4A, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.777740783 (329h12m44.239833256s) [noise=true , address=45.91.116.12:4225 , piece_id=E25URHWNFGSLPSILUVJ2RHBTNJJXCOBPGA4QUNWILXOXQJBA7EOA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.778119905 (329h12m44.240212378s) [noise=true , address=45.91.116.4:4938 , piece_id=AFFEFSCSNNVSJM2CR4E3IBURZLNTPVBK6OKT7DFDIMCERTDRZZKQ, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.778304958 (329h12m44.240397441s) [noise=true , address=45.91.116.8:4045 , piece_id=YMWHISCMJDM77HMQ2YC53PHSB4VVDZLRHFM6VKZZ57KFXOXD24UQ, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.778478018 (329h12m44.240570491s) [noise=true , address=45.91.116.13:4154 , piece_id=ILL6EJCPL7SY2XSFMJXP5MIXDU4NF3C3KQSZ5PR2NQHI3U7NP7CA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.778801704 (329h12m44.240894177s) [noise=true , address=45.91.116.13:4153 , piece_id=E4LY43VOF5V7SHSUXBVKSV5F45ZXBJHOR7AMWVF2TCRMLRIIKWUQ, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.778818706 (329h12m44.240911189s) [noise=true , address=77.90.31.12:4590 , piece_id=AWMYSILFUH5CMDJWWN5GXL3SPNLZ275C5E24QEJEBXOM2ULYAPWA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.779170577 (329h12m44.24126306s) [noise=true , address=145.79.227.2:4113 , piece_id=PVEQGLKGBCAEUJGUL6VFE52L4UUE7ILSFHILO3JOQFJZK6FLB7SA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.779187308 (329h12m44.241279791s) [noise=true , address=145.79.227.17:4971 , piece_id=JR2NFLPW47XLRMSCDIZIKEU7AAZBKQZ33PT2DTXCRDHIKCOP57PQ, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.779482330 (329h12m44.241574813s) [noise=true , address=45.91.116.14:4189 , piece_id=IHYLJ76HXBQFBR45ZWSKWXZYVFVQYFRQTQVR3JHW2APX3M774KZA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.779511706 (329h12m44.241604189s) [noise=true , address=77.90.31.18:4807 , piece_id=GSWCJUPEZ4ONGGXYO4VXEQCEIQ4E63LEB2AKZ5AHTNA44IKB3OGA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.779828439 (329h12m44.241920922s) [noise=true , address=45.91.116.5:4299 , piece_id=Z7HQ6G75EMHMPPWKFWJ6DRFLCDX57ZFHZNPLLKRJJ5IAJBNYJVYA, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:43:58.779950021 (329h12m44.242042504s) [noise=true , address=77.90.31.14:4661 , piece_id=NCTOVOPSFWAFW5YASBMS3M6WVLNORG3ITPTQE4VCZUSIIUZS7N5Q, seg_pos={0 0}, upload=289719]: Uploading piece... Oct 7 06:44:00.587873964 (329h12m46.049966447s) [upload=289719]: Next returned nil segment Oct 7 06:44:00.587900845 (329h12m46.049993319s) [upload=289719]: Waiting for error group managing segments... Oct 7 06:44:00.874825387 (329h12m46.33691787s) [noise=true , address=45.91.116.14:4189 , piece_id=IHYLJ76HXBQFBR45ZWSKWXZYVFVQYFRQTQVR3JHW2APX3M774KZA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.885259975 (329h12m46.347352459s) [noise=true , address=45.91.116.14:4191 , piece_id=TTM7JBIKPREG7YULIFAOCSC5BRS46IC2TSVL6UBRYMN4I7TSRT4A, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.909210071 (329h12m46.371302554s) [noise=true , address=45.91.116.4:4938 , piece_id=AFFEFSCSNNVSJM2CR4E3IBURZLNTPVBK6OKT7DFDIMCERTDRZZKQ, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.929126334 (329h12m46.391218817s) [noise=true , address=77.90.31.14:4661 , piece_id=NCTOVOPSFWAFW5YASBMS3M6WVLNORG3ITPTQE4VCZUSIIUZS7N5Q, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.929950434 (329h12m46.392042917s) [noise=true , address=45.91.116.8:4045 , piece_id=YMWHISCMJDM77HMQ2YC53PHSB4VVDZLRHFM6VKZZ57KFXOXD24UQ, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.943590587 (329h12m46.40568307s) [noise=true , address=77.90.31.10:4520 , piece_id=6K4DR5SSMJ74KZNLLTRLK2BM7HOMMO2RAOV7W5KC6F3UIAVAZ77A, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.965664921 (329h12m46.427757394s) [noise=true , address=45.91.116.13:4153 , piece_id=E4LY43VOF5V7SHSUXBVKSV5F45ZXBJHOR7AMWVF2TCRMLRIIKWUQ, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.976339757 (329h12m46.43843224s) [noise=true , address=45.91.116.12:4225 , piece_id=E25URHWNFGSLPSILUVJ2RHBTNJJXCOBPGA4QUNWILXOXQJBA7EOA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:00.979479024 (329h12m46.441571507s) [noise=true , address=45.91.116.5:4299 , piece_id=Z7HQ6G75EMHMPPWKFWJ6DRFLCDX57ZFHZNPLLKRJJ5IAJBNYJVYA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:01.004400198 (329h12m46.466492682s) [noise=true , address=77.90.31.12:4590 , piece_id=AWMYSILFUH5CMDJWWN5GXL3SPNLZ275C5E24QEJEBXOM2ULYAPWA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:01.008844041 (329h12m46.470936544s) [noise=true , address=77.90.31.18:4807 , piece_id=GSWCJUPEZ4ONGGXYO4VXEQCEIQ4E63LEB2AKZ5AHTNA44IKB3OGA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:01.041366699 (329h12m46.503459182s) [noise=true , address=45.91.116.13:4154 , piece_id=ILL6EJCPL7SY2XSFMJXP5MIXDU4NF3C3KQSZ5PR2NQHI3U7NP7CA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: <nil> Oct 7 06:44:27.828071449 (329h13m13.290163922s) [noise=true , address=145.79.227.4:4187 , piece_id=4PTDZ5SDFBUUNG4JZOJTUL4JC3CWYKAGAWDAC3S67JDNRMAHT3CQ, seg_pos={0 0}, upload=289719]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:55652->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:44:27.828264177 (329h13m13.29035665s) [noise=true , address=145.79.227.6:4258 , piece_id=IYRTJTFGWJD34TCTCK376NGUFJ2MLATVTB4HPSBX3R4JKSEP7YHA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: rpc: dial tcp 145.79.227.6:4258: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:44:27.828478395 (329h13m13.290570868s) [noise=true , address=145.79.227.2:4113 , piece_id=PVEQGLKGBCAEUJGUL6VFE52L4UUE7ILSFHILO3JOQFJZK6FLB7SA, seg_pos={0 0}, upload=289719]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: read tcp 188.166.225.170:56598->145.79.227.2:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:44:27.828763658 (329h13m13.290856141s) [noise=true , address=145.79.227.17:4971 , piece_id=JR2NFLPW47XLRMSCDIZIKEU7AAZBKQZ33PT2DTXCRDHIKCOP57PQ, seg_pos={0 0}, upload=289719]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:49410->145.79.227.17:4971: use of closed network connection; conn closed: op Oct 7 06:44:27.828822420 (329h13m13.290914893s) [seg_pos={0 0}, upload=289719]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 06:44:28.072050396 (329h13m13.534142869s) [upload=289719]: Done uploading segments... Oct 7 06:44:28.072077287 (329h13m13.53416976s) [upload=289719]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.246323582s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X....X] duration: 1m0.261243812s FAIL (2025-10-07 06:43:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== NJI3KJCCE3V6GXTNDHQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.397792889 (329h11m43.859885373s) [noise=true , address=64.49.13.137:4501 , piece_id=GIQQRHGX63IPQTWVXNXFA6K5Y7CWRB2KKMZ4M7PLX5KF2746CYTQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.397968223 (329h11m43.860060706s) [noise=true , address=77.90.31.17:4779 , piece_id=5LAWE5T66ZE3JY5CUHUOX5SGAI6KKNIUTLAQHDJCGBHXVVP26OGA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.398390818 (329h11m43.860483301s) [noise=true , address=77.90.31.10:4537 , piece_id=B3ZCOJTSIBYJNQHWSPM5DJOW7GDI6EV3UGBGB5ZSOOKSVNQI3Y7A, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.398515486 (329h11m43.860607969s) [noise=true , address=145.79.227.52:4445 , piece_id=DLH5MKIQFGTU22BYLNZ6FWD43IKU4OPKWV5EZ6CXJIPRA3MKOBRA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.398799106 (329h11m43.860891599s) [noise=true , address=145.79.227.35:4489 , piece_id=4PDSJV66E5KZYMZJH2ZXTBGHLSC7TA67OOTR5HKMMMDNFXXT6PNQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399021328 (329h11m43.861113801s) [noise=true , address=145.79.227.26:4662 , piece_id=A3SIDE35VMW3HOKLID5VJ2PHRRFKX7AIFYOBU3ZTRKBZE2YNRTDQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399207743 (329h11m43.861300226s) [noise=true , address=145.79.227.22:4529 , piece_id=QGHGSDVHQTVF5UGHPWRPPKO4ZLBVYBOCQ3G7MOTEQTA4EZVO2S2A, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399525268 (329h11m43.861617752s) [noise=true , address=145.79.227.44:4201 , piece_id=PSMY6DXTGJ5J6H7O6Y2EKGCESB6D6BDXQTO3US7LIHMZM4TKIE5A, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.399646208 (329h11m43.861738691s) [noise=true , address=77.90.31.19:4866 , piece_id=M3P6WWAVY2PO7F4GBASZVR5ET33DI5RQHFC3BUKQF6SVTZ3NOPLA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.400099010 (329h11m43.862191484s) [noise=true , address=77.90.31.15:4714 , piece_id=GDH2YXYYKUJAY2B7CKVFXDEB4MFLZLPCUXLL2O2DUKY5B45SQ3FA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.400147793 (329h11m43.862240276s) [noise=true , address=45.91.116.8:4069 , piece_id=INAZ2ZSSPJ5XKSY44IBMW36KHHQNKHTOUS5BO4SG5LURMVGWIZ7Q, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.401488466 (329h11m43.863580939s) [noise=true , address=77.90.31.18:4840 , piece_id=W4DPPNJQVCGBM7GHBKZKKR7DXNPQA4AW5ZIYOBD5S44PDHF3FPTA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.402290413 (329h11m43.864382896s) [noise=true , address=45.91.116.11:4124 , piece_id=UPOBMCYB6VMPSGQA2XOGPH5E3ACOG36I5SJ22C25TY4RQKOMWERA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.403080789 (329h11m43.865173272s) [noise=true , address=77.90.31.17:4780 , piece_id=7KNWVCIF45CTXKZ4ND52KNUOHD6S4RZ3OOAXJPVZY3IDRAUEQ7KA, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.404309278 (329h11m43.866401761s) [noise=true , address=145.79.227.20:4454 , piece_id=QC7DHFH27O5Y72LCLPH5PG3VWWDY3A2FMO3W6YKI2VHA74BPIUDQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:58.405692161 (329h11m43.867784654s) [noise=true , address=77.90.31.13:4641 , piece_id=PB2JQE5QB7DYGVMEZRMF4PBV6FHMTSDPDE6525H54WOJZMPYATIQ, seg_pos={0 0}, upload=289707]: Uploading piece... Oct 7 06:42:59.949497860 (329h11m45.411590343s) [upload=289707]: Next returned nil segment Oct 7 06:42:59.949522658 (329h11m45.411615151s) [upload=289707]: Waiting for error group managing segments... Oct 7 06:43:00.339907244 (329h11m45.801999727s) [noise=true , address=77.90.31.17:4780 , piece_id=7KNWVCIF45CTXKZ4ND52KNUOHD6S4RZ3OOAXJPVZY3IDRAUEQ7KA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.366592848 (329h11m45.828685331s) [noise=true , address=77.90.31.13:4641 , piece_id=PB2JQE5QB7DYGVMEZRMF4PBV6FHMTSDPDE6525H54WOJZMPYATIQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.373948594 (329h11m45.836041077s) [noise=true , address=45.91.116.8:4069 , piece_id=INAZ2ZSSPJ5XKSY44IBMW36KHHQNKHTOUS5BO4SG5LURMVGWIZ7Q, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.382656756 (329h11m45.844749239s) [noise=true , address=77.90.31.10:4537 , piece_id=B3ZCOJTSIBYJNQHWSPM5DJOW7GDI6EV3UGBGB5ZSOOKSVNQI3Y7A, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.394420667 (329h11m45.85651315s) [noise=true , address=77.90.31.18:4840 , piece_id=W4DPPNJQVCGBM7GHBKZKKR7DXNPQA4AW5ZIYOBD5S44PDHF3FPTA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.402582077 (329h11m45.86467456s) [noise=true , address=77.90.31.15:4714 , piece_id=GDH2YXYYKUJAY2B7CKVFXDEB4MFLZLPCUXLL2O2DUKY5B45SQ3FA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.430546977 (329h11m45.89263946s) [noise=true , address=77.90.31.19:4866 , piece_id=M3P6WWAVY2PO7F4GBASZVR5ET33DI5RQHFC3BUKQF6SVTZ3NOPLA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.433192705 (329h11m45.895285188s) [noise=true , address=77.90.31.14:4689 , piece_id=4H2XAFHUCU4V4BIYU3UKFVG54R736QH3PS4BA727EY76VWA6JK2Q, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.493973121 (329h11m45.956065604s) [noise=true , address=45.91.116.11:4124 , piece_id=UPOBMCYB6VMPSGQA2XOGPH5E3ACOG36I5SJ22C25TY4RQKOMWERA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:00.559588964 (329h11m46.021681447s) [noise=true , address=77.90.31.17:4779 , piece_id=5LAWE5T66ZE3JY5CUHUOX5SGAI6KKNIUTLAQHDJCGBHXVVP26OGA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: <nil> Oct 7 06:43:57.423779761 (329h12m42.885872244s) [noise=true , address=145.79.227.44:4201 , piece_id=PSMY6DXTGJ5J6H7O6Y2EKGCESB6D6BDXQTO3US7LIHMZM4TKIE5A, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12bPDvhDfu8Z58LxZcicg1vAiiu1sZYGpwQnvuhk4t5m6yR87Fi, address:145.79.227.44:4201): protocol: rpc: dial tcp 145.79.227.44:4201: i/o timeout; context deadline exceeded Oct 7 06:43:57.423882767 (329h12m42.88597524s) [noise=true , address=145.79.227.22:4529 , piece_id=QGHGSDVHQTVF5UGHPWRPPKO4ZLBVYBOCQ3G7MOTEQTA4EZVO2S2A, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12bNgTA4CY9ShB6Mi3QpceTEqNCr8ohCMHNVvU6S1qJp2TdVP1i, address:145.79.227.22:4529): protocol: read tcp 188.166.225.170:55814->145.79.227.22:4529: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.423959603 (329h12m42.886052076s) [noise=true , address=64.49.13.140:4513 , piece_id=LYF2S6TEPKQ3M6HGCKGCPCXIWVQOYZU56NJI3KJCCE3V6GXTNDHQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:1Ac3N4hMJ66bTx2dpHBDT6W5SEGnPNfwyonwmvAANR8mM8vvpY, address:64.49.13.140:4513): protocol: write tcp 188.166.225.170:53248->64.49.13.140:4513: use of closed network connection; write tcp 188.166.225.170:53248->64.49.13.140:4513: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53248->64.49.13.140:4513: use of closed network connection Oct 7 06:43:57.424030448 (329h12m42.886122921s) [noise=true , address=145.79.227.52:4445 , piece_id=DLH5MKIQFGTU22BYLNZ6FWD43IKU4OPKWV5EZ6CXJIPRA3MKOBRA, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12vXrkhLqCofhR4Fmfqb98gxnqWQF6vMV4q3ATbRvPfcqi82WSo, address:145.79.227.52:4445): protocol: rpc: dial tcp 145.79.227.52:4445: i/o timeout; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 06:43:57.424049665 (329h12m42.886142148s) [noise=true , address=64.49.13.137:4501 , piece_id=GIQQRHGX63IPQTWVXNXFA6K5Y7CWRB2KKMZ4M7PLX5KF2746CYTQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12D2MWhZKXEMUPBvMQVfzVJZNw4m857MYZ4BjLfEwVxS4fJLzj1, address:64.49.13.137:4501): protocol: read tcp 188.166.225.170:42886->64.49.13.137:4501: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.424269313 (329h12m42.886361796s) [noise=true , address=145.79.227.26:4662 , piece_id=A3SIDE35VMW3HOKLID5VJ2PHRRFKX7AIFYOBU3ZTRKBZE2YNRTDQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:1uyMAEh41FVYSVGkNjzweDWqHKaFzY9YvxKcCFJTJ4Y5cehKZV, address:145.79.227.26:4662): protocol: read tcp 188.166.225.170:60434->145.79.227.26:4662: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.424306664 (329h12m42.886399147s) [noise=true , address=145.79.227.35:4489 , piece_id=4PDSJV66E5KZYMZJH2ZXTBGHLSC7TA67OOTR5HKMMMDNFXXT6PNQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:12YbmXQeh182PYH1AHxvGcw3zW3uiuDsbYPomH3HAPNtjbJSpPw, address:145.79.227.35:4489): protocol: read tcp 188.166.225.170:32942->145.79.227.35:4489: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:57.424372870 (329h12m42.886465353s) [noise=true , address=145.79.227.20:4454 , piece_id=QC7DHFH27O5Y72LCLPH5PG3VWWDY3A2FMO3W6YKI2VHA74BPIUDQ, seg_pos={0 0}, upload=289707]: Done uploading piece. err: ecclient: upload failed (node:1SafkJPqKHLwMaqyZdLq3yJ1xdjqDtWHFrUwynmegUVwEC7y5J, address:145.79.227.20:4454): protocol: context deadline exceeded; context deadline exceeded Oct 7 06:43:57.424446831 (329h12m42.886539314s) [seg_pos={0 0}, upload=289707]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Oct 7 06:43:57.682720297 (329h12m43.14481279s) [upload=289707]: Done uploading segments... Oct 7 06:43:57.682759371 (329h12m43.144851864s) [upload=289707]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.261275842s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [....................................................X.....XX] duration: 30.248069823s FAIL (2025-10-07 06:43:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 06:42:53.450398502 (329h11m38.912490985s) [upload=289705]: Starting upload Oct 7 06:42:53.450573024 (329h11m38.912665497s) [upload=289705]: Done starting upload Oct 7 06:42:53.450702742 (329h11m38.912795215s) [upload=289705]: Uploading segments... Oct 7 06:42:53.450875259 (329h11m38.912967732s) [upload=289705]: Got next segment. Inline: false Oct 7 06:42:53.450914445 (329h11m38.913006918s) [upload=289705]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:42:53.450919785 (329h11m38.913012258s) [upload=289705]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:42:53.797571906 (329h11m39.259664379s) [seg_pos={0 0}, upload=289705]: Begin upload segment... Oct 7 06:42:53.797621541 (329h11m39.259714024s) [seg_pos={0 0}, upload=289705]: Done begin upload segment. Oct 7 06:42:53.797649054 (329h11m39.259741537s) [noise=true , address=145.79.227.6:4259 , piece_id=WRWUEAZQTXG6OMAYEZNQTZYXD5WBCTQVZZWAZVHBU7YHBKIV77QQ, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.798235870 (329h11m39.260328353s) [noise=true , address=77.90.31.11:4553 , piece_id=EEAI272F4T3NFMUDJHFO3VEXFBYEZGLY4H75CCAGN5WJX26NJVPA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.798820934 (329h11m39.260913417s) [noise=true , address=145.79.227.3:4150 , piece_id=GZGDZ7DPH6LUKCJXMHWM6LSRVUDVGCZ4AG6GUCXEWTMO4FKBQYFQ, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.799196159 (329h11m39.261288652s) [noise=true , address=77.90.31.13:4626 , piece_id=GS6HCKUBV5MLJ7ADGD7C422H64U6KVXOIHQS5KY6ML6X5LUNATNA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.799806682 (329h11m39.261899165s) [noise=true , address=77.90.31.15:4697 , piece_id=F6KAKWT7S4N726A2QJPEBMJOC374CERXKZNIHUMG5RK5UE6DVL4A, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.800264975 (329h11m39.262357458s) [noise=true , address=77.90.31.16:4733 , piece_id=3DS434OKKUFRLYPE46UR4ELXMZYAPYBLSFEP545FK7I62FB7VK4A, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.800790235 (329h11m39.262882718s) [noise=true , address=45.91.116.7:4009 , piece_id=GMHW2WKXW3HLPHU7HBIMGLF4SEOY5UW6SQJ6TN2CLRYHKI5VYXNA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.802031529 (329h11m39.264124012s) [noise=true , address=145.79.227.18:4007 , piece_id=DJRVAAFKXXL3AMW4CALJEVYZXRMIDVTC5HRYVJ4GKA653WMLJFBA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.802240166 (329h11m39.264332649s) [noise=true , address=145.79.227.2:4113 , piece_id=23KSXZV7ETRGVAEY46JPSKDQFCZTZRBHK3MF2UUIBQLYQ34PQHWQ, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.803975209 (329h11m39.266067692s) [noise=true , address=45.91.116.12:4226 , piece_id=HZIH3J4HHEL6SLPD6EKII4QK3CNXYHHCUJIJMVVTJEDV3O5IEVGA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.804186481 (329h11m39.266278964s) [noise=true , address=145.79.227.4:4187 , piece_id=RECRFWQLGE3RYQ7FSO2QZ4TPCJXTGDNR4MV4VCVIQKG33AHR7BGA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.805602358 (329h11m39.267694841s) [noise=true , address=45.91.116.8:4047 , piece_id=7WEMKHFZKYGZMXZDZAECF2N7S7UTE6ZWDR3ECBHELGCCGEXDJ3MA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.806122628 (329h11m39.268215111s) [noise=true , address=77.90.31.12:4592 , piece_id=LVS3NV7T2LNAJ42CKHU7WFG7AJLL6BQC27E6MRVQN3TJK3QZOGJQ, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.807931934 (329h11m39.270024407s) [noise=true , address=45.91.116.9:4262 , piece_id=LYZPZQT5Q3KOSBLAVU3A7OMAWXPQRXVYRTPHBPMAQG4YKZGSTHGA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.809883259 (329h11m39.271975742s) [noise=true , address=45.91.116.14:4191 , piece_id=BKDTZFQUSMUU3AFKBROIU76FFOSCRYFW3MAUFQNVAODYBV6DD7YA, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:53.810659908 (329h11m39.272752391s) [noise=true , address=77.90.31.17:4772 , piece_id=P4DGNE2ARRBV3WL5UFK3AKPFN7RD7BCCCMUDW33IS65VNNL3V4SQ, seg_pos={0 0}, upload=289705]: Uploading piece... Oct 7 06:42:55.585911940 (329h11m41.048004423s) [upload=289705]: Next returned nil segment Oct 7 06:42:55.585931106 (329h11m41.048023589s) [upload=289705]: Waiting for error group managing segments... Oct 7 06:42:55.866667985 (329h11m41.328760458s) [noise=true , address=45.91.116.9:4262 , piece_id=LYZPZQT5Q3KOSBLAVU3A7OMAWXPQRXVYRTPHBPMAQG4YKZGSTHGA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:55.906815713 (329h11m41.368908186s) [noise=true , address=45.91.116.7:4009 , piece_id=GMHW2WKXW3HLPHU7HBIMGLF4SEOY5UW6SQJ6TN2CLRYHKI5VYXNA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:55.957295134 (329h11m41.419387607s) [noise=true , address=45.91.116.12:4226 , piece_id=HZIH3J4HHEL6SLPD6EKII4QK3CNXYHHCUJIJMVVTJEDV3O5IEVGA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:55.964280234 (329h11m41.426372717s) [noise=true , address=77.90.31.15:4697 , piece_id=F6KAKWT7S4N726A2QJPEBMJOC374CERXKZNIHUMG5RK5UE6DVL4A, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:55.982993388 (329h11m41.445085871s) [noise=true , address=45.91.116.8:4047 , piece_id=7WEMKHFZKYGZMXZDZAECF2N7S7UTE6ZWDR3ECBHELGCCGEXDJ3MA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:55.995749006 (329h11m41.457841499s) [noise=true , address=77.90.31.17:4772 , piece_id=P4DGNE2ARRBV3WL5UFK3AKPFN7RD7BCCCMUDW33IS65VNNL3V4SQ, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:55.999005007 (329h11m41.46109749s) [noise=true , address=77.90.31.13:4626 , piece_id=GS6HCKUBV5MLJ7ADGD7C422H64U6KVXOIHQS5KY6ML6X5LUNATNA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:56.003409763 (329h11m41.465502277s) [noise=true , address=45.91.116.14:4191 , piece_id=BKDTZFQUSMUU3AFKBROIU76FFOSCRYFW3MAUFQNVAODYBV6DD7YA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:56.048612206 (329h11m41.510704679s) [noise=true , address=77.90.31.12:4592 , piece_id=LVS3NV7T2LNAJ42CKHU7WFG7AJLL6BQC27E6MRVQN3TJK3QZOGJQ, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:56.370879176 (329h11m41.83297166s) [noise=true , address=77.90.31.11:4553 , piece_id=EEAI272F4T3NFMUDJHFO3VEXFBYEZGLY4H75CCAGN5WJX26NJVPA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:42:58.746745353 (329h11m44.208837836s) [noise=true , address=77.90.31.16:4733 , piece_id=3DS434OKKUFRLYPE46UR4ELXMZYAPYBLSFEP545FK7I62FB7VK4A, seg_pos={0 0}, upload=289705]: Done uploading piece. err: <nil> Oct 7 06:43:22.739147371 (329h12m8.201239854s) [noise=true , address=145.79.227.4:4187 , piece_id=RECRFWQLGE3RYQ7FSO2QZ4TPCJXTGDNR4MV4VCVIQKG33AHR7BGA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: rpc: dial tcp 145.79.227.4:4187: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:22.739390123 (329h12m8.201482596s) [noise=true , address=145.79.227.2:4113 , piece_id=23KSXZV7ETRGVAEY46JPSKDQFCZTZRBHK3MF2UUIBQLYQ34PQHWQ, seg_pos={0 0}, upload=289705]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: rpc: dial tcp 145.79.227.2:4113: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:22.739561579 (329h12m8.201654062s) [noise=true , address=145.79.227.3:4150 , piece_id=GZGDZ7DPH6LUKCJXMHWM6LSRVUDVGCZ4AG6GUCXEWTMO4FKBQYFQ, seg_pos={0 0}, upload=289705]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: rpc: dial tcp 145.79.227.3:4150: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:22.739792669 (329h12m8.201885152s) [noise=true , address=145.79.227.18:4007 , piece_id=DJRVAAFKXXL3AMW4CALJEVYZXRMIDVTC5HRYVJ4GKA653WMLJFBA, seg_pos={0 0}, upload=289705]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:41726->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:22.739950500 (329h12m8.202042983s) [noise=true , address=145.79.227.6:4259 , piece_id=WRWUEAZQTXG6OMAYEZNQTZYXD5WBCTQVZZWAZVHBU7YHBKIV77QQ, seg_pos={0 0}, upload=289705]: Done uploading piece. err: ecclient: upload failed (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y, address:145.79.227.6:4259): protocol: read tcp 188.166.225.170:59620->145.79.227.6:4259: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:43:22.740022797 (329h12m8.20211528s) [seg_pos={0 0}, upload=289705]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 06:43:22.985436355 (329h12m8.447528838s) [upload=289705]: Done uploading segments... Oct 7 06:43:22.985472203 (329h12m8.447564687s) [upload=289705]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.248088859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.....X] duration: 30.247366202s FAIL (2025-10-07 06:42:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Oct 7 06:42:22.967244391 (329h11m8.429336864s) [upload=289700]: Starting upload Oct 7 06:42:22.967331076 (329h11m8.429423559s) [upload=289700]: Done starting upload Oct 7 06:42:22.967466343 (329h11m8.429558816s) [upload=289700]: Uploading segments... Oct 7 06:42:22.967643661 (329h11m8.429736144s) [upload=289700]: Got next segment. Inline: false Oct 7 06:42:22.967688106 (329h11m8.429780589s) [upload=289700]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:42:22.967712572 (329h11m8.429805055s) [upload=289700]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:42:23.297516525 (329h11m8.759609018s) [seg_pos={0 0}, upload=289700]: Begin upload segment... Oct 7 06:42:23.297602518 (329h11m8.759695001s) [seg_pos={0 0}, upload=289700]: Done begin upload segment. Oct 7 06:42:23.297639158 (329h11m8.759731631s) [noise=true , address=145.79.227.7:4295 , piece_id=B4URPZAYENH7BSPJUC6SXJLE6R2S33BFKLPXDTKTBYJ5GW6D5SFQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.297836424 (329h11m8.759928907s) [noise=true , address=145.79.227.16:4911 , piece_id=MDY2RPUTZPNSDSTM2RAAK22ROTO4LMNWVI7CHTQECRP7XLYXNLYQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.298479207 (329h11m8.76057172s) [noise=true , address=45.91.116.8:4047 , piece_id=4NLX5U6MKEOAQI7IJ2DEDXNYVYK6QWTIO3SVKJD756RXANNVDAIA, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.298962648 (329h11m8.761055131s) [noise=true , address=77.90.31.10:4519 , piece_id=YEJGPS3HP3B6F2YIG5ZMIBYSASFSNCLAH5FXB7KA2GNFF726JGZQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.299330128 (329h11m8.761422601s) [noise=true , address=77.90.31.12:4590 , piece_id=DVAUYAIGZ4VAKCYA3DPGZUB54JCLPN3F7IN7MSGOBMN4XDFQFYAA, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.299729067 (329h11m8.76182154s) [noise=true , address=77.90.31.11:4554 , piece_id=LNR6CPL6CSZH5HDZYPSNAN3UJICX4IFGBZ3ITY34MILAXMDFB7HQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.300139970 (329h11m8.762232453s) [noise=true , address=145.79.227.7:4293 , piece_id=OMXUCVRU3OLWXUTINROXBKJR7CQGC3HIYQ3ELFVW2OOMMJ6OSGZQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.300513932 (329h11m8.762606405s) [noise=true , address=145.79.227.5:4223 , piece_id=ZPA6ZRPCNOUZDOCQYX2NB5K34ZCX6LRJCKLFDQRMJRERE4635EBQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.301012441 (329h11m8.763104924s) [noise=true , address=45.91.116.12:4226 , piece_id=7FB2SNJWPBMZ7MCZTU3NWKR3EQYGRLAOJGWAYTTVUDIMY6WA2T7Q, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.302739420 (329h11m8.764831893s) [noise=true , address=77.90.31.14:4661 , piece_id=TEYH34573N43TBCWLA7POTVCD5YGABUSAT4SU4GU7KIMFU2B25YA, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.304382228 (329h11m8.766474711s) [noise=true , address=77.90.31.18:4807 , piece_id=LFYETYVYML5M5PW2HQT4CSOM3CPVCZGHTI4FIJ263SMPKUM5YHPQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.306429025 (329h11m8.768521509s) [noise=true , address=145.79.227.4:4186 , piece_id=KGEN7BUP7YFARWJGSTQYNEKBRM63IVMGBGONHZ5CXJQ6YWDOKWLQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.307928140 (329h11m8.770020623s) [noise=true , address=77.90.31.17:4770 , piece_id=GLIBLC7XLXEZXI73AKUG4VJVTGVQELVVLE6FQZAXIWFDFUAUVI6Q, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.309290294 (329h11m8.771382767s) [noise=true , address=145.79.227.6:4258 , piece_id=OKHP3PCYSJYIE4D2YLMUIEEJXQVBYHNF4YV2EQNRQTB56KVFLRNQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.310657156 (329h11m8.772749629s) [noise=true , address=145.79.227.14:4837 , piece_id=X5KALZZIBRPWVYCAPPLLCQ3TYVD7DJXUY7DDDTFMZPJ4DJWYHQNQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:23.311904642 (329h11m8.773997115s) [noise=true , address=45.91.116.7:4009 , piece_id=7P4PKQJVKA4VAS6XVCTGCABB3L5TI4ZZPCSX7PU5AT4KRKENUZXQ, seg_pos={0 0}, upload=289700]: Uploading piece... Oct 7 06:42:25.105603647 (329h11m10.56769612s) [upload=289700]: Next returned nil segment Oct 7 06:42:25.105630238 (329h11m10.567722711s) [upload=289700]: Waiting for error group managing segments... Oct 7 06:42:25.394500925 (329h11m10.856593418s) [noise=true , address=77.90.31.17:4770 , piece_id=GLIBLC7XLXEZXI73AKUG4VJVTGVQELVVLE6FQZAXIWFDFUAUVI6Q, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.429481724 (329h11m10.891574217s) [noise=true , address=45.91.116.12:4226 , piece_id=7FB2SNJWPBMZ7MCZTU3NWKR3EQYGRLAOJGWAYTTVUDIMY6WA2T7Q, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.437120751 (329h11m10.899213234s) [noise=true , address=77.90.31.18:4807 , piece_id=LFYETYVYML5M5PW2HQT4CSOM3CPVCZGHTI4FIJ263SMPKUM5YHPQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.467154559 (329h11m10.929247042s) [noise=true , address=45.91.116.7:4009 , piece_id=7P4PKQJVKA4VAS6XVCTGCABB3L5TI4ZZPCSX7PU5AT4KRKENUZXQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.480768132 (329h11m10.942860615s) [noise=true , address=77.90.31.11:4554 , piece_id=LNR6CPL6CSZH5HDZYPSNAN3UJICX4IFGBZ3ITY34MILAXMDFB7HQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.483629440 (329h11m10.945721913s) [noise=true , address=77.90.31.12:4590 , piece_id=DVAUYAIGZ4VAKCYA3DPGZUB54JCLPN3F7IN7MSGOBMN4XDFQFYAA, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.491332228 (329h11m10.953424711s) [noise=true , address=77.90.31.14:4661 , piece_id=TEYH34573N43TBCWLA7POTVCD5YGABUSAT4SU4GU7KIMFU2B25YA, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:25.509875287 (329h11m10.97196777s) [noise=true , address=77.90.31.10:4519 , piece_id=YEJGPS3HP3B6F2YIG5ZMIBYSASFSNCLAH5FXB7KA2GNFF726JGZQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:26.356827573 (329h11m11.818920066s) [noise=true , address=45.91.116.8:4047 , piece_id=4NLX5U6MKEOAQI7IJ2DEDXNYVYK6QWTIO3SVKJD756RXANNVDAIA, seg_pos={0 0}, upload=289700]: Done uploading piece. err: <nil> Oct 7 06:42:52.274149021 (329h11m37.736241504s) [noise=true , address=145.79.227.7:4293 , piece_id=OMXUCVRU3OLWXUTINROXBKJR7CQGC3HIYQ3ELFVW2OOMMJ6OSGZQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: rpc: dial tcp 145.79.227.7:4293: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274271735 (329h11m37.736364208s) [noise=true , address=145.79.227.7:4295 , piece_id=B4URPZAYENH7BSPJUC6SXJLE6R2S33BFKLPXDTKTBYJ5GW6D5SFQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:56832->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274307743 (329h11m37.736400216s) [noise=true , address=145.79.227.14:4837 , piece_id=X5KALZZIBRPWVYCAPPLLCQ3TYVD7DJXUY7DDDTFMZPJ4DJWYHQNQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: rpc: dial tcp 145.79.227.14:4837: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274398005 (329h11m37.736490488s) [noise=true , address=145.79.227.5:4223 , piece_id=ZPA6ZRPCNOUZDOCQYX2NB5K34ZCX6LRJCKLFDQRMJRERE4635EBQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: read tcp 188.166.225.170:34184->145.79.227.5:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274469101 (329h11m37.736561574s) [noise=true , address=145.79.227.16:4911 , piece_id=MDY2RPUTZPNSDSTM2RAAK22ROTO4LMNWVI7CHTQECRP7XLYXNLYQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 188.166.225.170:60438->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274569091 (329h11m37.736661564s) [noise=true , address=145.79.227.4:4186 , piece_id=KGEN7BUP7YFARWJGSTQYNEKBRM63IVMGBGONHZ5CXJQ6YWDOKWLQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: read tcp 188.166.225.170:60990->145.79.227.4:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274610270 (329h11m37.736702743s) [noise=true , address=145.79.227.6:4258 , piece_id=OKHP3PCYSJYIE4D2YLMUIEEJXQVBYHNF4YV2EQNRQTB56KVFLRNQ, seg_pos={0 0}, upload=289700]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:58484->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 06:42:52.274638463 (329h11m37.736730946s) [seg_pos={0 0}, upload=289700]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 06:42:52.519137659 (329h11m37.981230132s) [upload=289700]: Done uploading segments... Oct 7 06:42:52.519174619 (329h11m37.981267092s) [upload=289700]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.247390107s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 5.838391373s FAIL (2025-10-07 06:41:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, Piece ID: 3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA) context deadline exceeded; piecestore: (Node ID: 123r9xrR18b29rsRY6APtib52WagU7ttLA7MHY4EQ7inh6BjdCB, Piece ID: HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ) context deadline exceeded; piecestore: (Node ID: 12wXqePK1kK7iBo6wHcbqkRmkED8LGJ3rzQNJAWa14cHYhx1oyB, Piece ID: NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A) context deadline exceeded; piecestore: (Node ID: 12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg, Piece ID: NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A) context deadline exceeded ======LOGS====== pload=289637]: Uploading piece... Oct 7 06:38:06.420968208 (329h6m51.883060691s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.421700603 (329h6m51.883793076s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422059165 (329h6m51.884151638s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422378814 (329h6m51.884471297s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422666322 (329h6m51.884758795s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422953418 (329h6m51.885045901s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423269190 (329h6m51.885361663s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423585622 (329h6m51.885678105s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423893889 (329h6m51.885986362s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424305283 (329h6m51.886397766s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424825754 (329h6m51.886918237s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425352837 (329h6m51.88744532s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425963780 (329h6m51.888056263s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426564014 (329h6m51.888656497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426685595 (329h6m51.888778088s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.427260630 (329h6m51.889353113s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.428079549 (329h6m51.890172032s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:07.867445156 (329h6m53.329537639s) [upload=289637]: Next returned nil segment Oct 7 06:38:07.867467089 (329h6m53.329559572s) [upload=289637]: Waiting for error group managing segments... Oct 7 06:38:08.257086538 (329h6m53.719179021s) [noise=true , address=64.49.13.138:4481 , piece_id=JO4TMBHCZZ7DQMRZQJPJFR52PBJTIZ54Q5LKATJ5ORDTWG52YKMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.336818631 (329h6m53.798911124s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.366270112 (329h6m53.828362595s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.397675292 (329h6m53.859767775s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.400627513 (329h6m53.862719996s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.414548000 (329h6m53.876640483s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.467298877 (329h6m53.92939136s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.490839973 (329h6m53.952932456s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.524333590 (329h6m53.986426073s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.534156615 (329h6m53.996249088s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:14.068606852 (329h6m59.530699325s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:17.034284555 (329h7m2.496377038s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:21.896577969 (329h7m7.358670452s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417662014 (329h7m10.879754497s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417702531 (329h7m10.879795014s) [seg_pos={0 0}, upload=289637]: Segment reached optimal threshold of 14 pieces. Oct 7 06:38:25.417898014 (329h7m10.879990497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FJGmqUskGmxYctVCEaYhRHq98m1VCVZiWL1ShEMJzmieaYo1u): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417911219 (329h7m10.880003702s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.417976213 (329h7m10.880068696s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1asDxxBjiU6yNbZ5c6uBmSWQ4QbVs5J3ErLbpKBSwqaptpNhpn): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417994398 (329h7m10.880086881s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418029004 (329h7m10.880121487s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12chhfe1weiPC1jjv8vbScP7uFVNXFr6s3UcNRkH8gsSAtGD16D): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418038491 (329h7m10.880130964s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418064501 (329h7m10.880156974s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jvSGi6s5yi55BBFJ4aErrap64wGP2WbDL9oUB6pdGiv8DM385): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418070512 (329h7m10.880162986s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418088236 (329h7m10.880180719s) [seg_pos={0 0}, upload=289637]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:38:25.418177306 (329h7m10.880269779s) [upload=289637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:38:25.418182626 (329h7m10.880275099s) [upload=289637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:38:25.730712968 (329h7m11.192805452s) [upload=289637]: Done uploading segments... Oct 7 06:38:25.730733146 (329h7m11.192825619s) [upload=289637]: Upload finished. err: <nil> Duration: 5.838423524s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.246361425s FAIL (2025-10-07 06:41:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: 2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA) context deadline exceeded; piecestore: (Node ID: 12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, Piece ID: LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ) context deadline exceeded; piecestore: (Node ID: 12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, Piece ID: 7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA) context deadline exceeded; piecestore: (Node ID: 12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, Piece ID: MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q) context deadline exceeded; piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ) context deadline exceeded ======LOGS====== 38:51.525809865 (329h7m36.987902338s) [upload=289645]: Uploading segments... Oct 7 06:38:51.526041425 (329h7m36.988133908s) [upload=289645]: Got next segment. Inline: false Oct 7 06:38:51.526090008 (329h7m36.988182501s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:38:51.526096420 (329h7m36.988188893s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:38:51.786641821 (329h7m37.248734294s) [seg_pos={0 0}, upload=289645]: Begin upload segment... Oct 7 06:38:51.786704200 (329h7m37.248796673s) [seg_pos={0 0}, upload=289645]: Done begin upload segment. Oct 7 06:38:51.786748324 (329h7m37.248840807s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.786761349 (329h7m37.248853822s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787235772 (329h7m37.249328245s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787646174 (329h7m37.249738657s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.788493227 (329h7m37.25058571s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789213488 (329h7m37.251305971s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789676100 (329h7m37.251768583s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789893954 (329h7m37.251986437s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790205708 (329h7m37.252298191s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790776945 (329h7m37.252869428s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.792121125 (329h7m37.254213608s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.793644276 (329h7m37.255736759s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.795373438 (329h7m37.257465921s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796885548 (329h7m37.258978041s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796913079 (329h7m37.259005552s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.797359229 (329h7m37.259451712s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:53.629515485 (329h7m39.091607968s) [upload=289645]: Next returned nil segment Oct 7 06:38:53.629538268 (329h7m39.091630751s) [upload=289645]: Waiting for error group managing segments... Oct 7 06:38:53.918072374 (329h7m39.380164847s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.940442060 (329h7m39.402534543s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.964069099 (329h7m39.426161582s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.996913059 (329h7m39.459005542s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.999517939 (329h7m39.461610412s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:54.107690628 (329h7m39.569783112s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:14.627829239 (329h8m0.089921712s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.614758925 (329h8m1.076851408s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.670324441 (329h8m1.132416924s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.926280263 (329h8m1.388372756s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.605972466 (329h8m4.068064949s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.778892774 (329h8m4.240985257s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922160720 (329h8m4.384253203s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922200937 (329h8m4.38429343s) [seg_pos={0 0}, upload=289645]: Segment reached optimal threshold of 13 pieces. Oct 7 06:39:18.922491700 (329h8m4.384584183s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection Oct 7 06:39:18.922504335 (329h8m4.384596818s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922631556 (329h8m4.384724039s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection Oct 7 06:39:18.922650843 (329h8m4.384743317s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922662095 (329h8m4.384754578s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: context canceled; context canceled Oct 7 06:39:18.922670070 (329h8m4.384762543s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922688044 (329h8m4.384780527s) [seg_pos={0 0}, upload=289645]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:39:18.922800388 (329h8m4.384892861s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:39:18.922805427 (329h8m4.3848979s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:39:19.252191662 (329h8m4.714284146s) [upload=289645]: Done uploading segments... Oct 7 06:39:19.252213765 (329h8m4.714306238s) [upload=289645]: Upload finished. err: <nil> Duration: 6.246392704s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.124826902s FAIL (2025-10-07 06:41:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: 2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA) context deadline exceeded; piecestore: (Node ID: 12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, Piece ID: LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ) context deadline exceeded; piecestore: (Node ID: 12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, Piece ID: 7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA) context deadline exceeded; piecestore: (Node ID: 12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, Piece ID: MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q) context deadline exceeded; piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ) context deadline exceeded ======LOGS====== 38:51.525809865 (329h7m36.987902338s) [upload=289645]: Uploading segments... Oct 7 06:38:51.526041425 (329h7m36.988133908s) [upload=289645]: Got next segment. Inline: false Oct 7 06:38:51.526090008 (329h7m36.988182501s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:38:51.526096420 (329h7m36.988188893s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:38:51.786641821 (329h7m37.248734294s) [seg_pos={0 0}, upload=289645]: Begin upload segment... Oct 7 06:38:51.786704200 (329h7m37.248796673s) [seg_pos={0 0}, upload=289645]: Done begin upload segment. Oct 7 06:38:51.786748324 (329h7m37.248840807s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.786761349 (329h7m37.248853822s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787235772 (329h7m37.249328245s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787646174 (329h7m37.249738657s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.788493227 (329h7m37.25058571s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789213488 (329h7m37.251305971s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789676100 (329h7m37.251768583s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789893954 (329h7m37.251986437s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790205708 (329h7m37.252298191s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790776945 (329h7m37.252869428s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.792121125 (329h7m37.254213608s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.793644276 (329h7m37.255736759s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.795373438 (329h7m37.257465921s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796885548 (329h7m37.258978041s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796913079 (329h7m37.259005552s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.797359229 (329h7m37.259451712s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:53.629515485 (329h7m39.091607968s) [upload=289645]: Next returned nil segment Oct 7 06:38:53.629538268 (329h7m39.091630751s) [upload=289645]: Waiting for error group managing segments... Oct 7 06:38:53.918072374 (329h7m39.380164847s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.940442060 (329h7m39.402534543s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.964069099 (329h7m39.426161582s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.996913059 (329h7m39.459005542s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.999517939 (329h7m39.461610412s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:54.107690628 (329h7m39.569783112s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:14.627829239 (329h8m0.089921712s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.614758925 (329h8m1.076851408s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.670324441 (329h8m1.132416924s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.926280263 (329h8m1.388372756s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.605972466 (329h8m4.068064949s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.778892774 (329h8m4.240985257s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922160720 (329h8m4.384253203s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922200937 (329h8m4.38429343s) [seg_pos={0 0}, upload=289645]: Segment reached optimal threshold of 13 pieces. Oct 7 06:39:18.922491700 (329h8m4.384584183s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection Oct 7 06:39:18.922504335 (329h8m4.384596818s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922631556 (329h8m4.384724039s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection Oct 7 06:39:18.922650843 (329h8m4.384743317s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922662095 (329h8m4.384754578s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: context canceled; context canceled Oct 7 06:39:18.922670070 (329h8m4.384762543s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922688044 (329h8m4.384780527s) [seg_pos={0 0}, upload=289645]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:39:18.922800388 (329h8m4.384892861s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:39:18.922805427 (329h8m4.3848979s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:39:19.252191662 (329h8m4.714284146s) [upload=289645]: Done uploading segments... Oct 7 06:39:19.252213765 (329h8m4.714306238s) [upload=289645]: Upload finished. err: <nil> Duration: 6.124856939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.92326227s FAIL (2025-10-07 06:41:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, Piece ID: 3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA) context deadline exceeded; piecestore: (Node ID: 123r9xrR18b29rsRY6APtib52WagU7ttLA7MHY4EQ7inh6BjdCB, Piece ID: HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ) context deadline exceeded; piecestore: (Node ID: 12wXqePK1kK7iBo6wHcbqkRmkED8LGJ3rzQNJAWa14cHYhx1oyB, Piece ID: NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A) context deadline exceeded; piecestore: (Node ID: 12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg, Piece ID: NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A) context deadline exceeded ======LOGS====== pload=289637]: Uploading piece... Oct 7 06:38:06.420968208 (329h6m51.883060691s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.421700603 (329h6m51.883793076s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422059165 (329h6m51.884151638s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422378814 (329h6m51.884471297s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422666322 (329h6m51.884758795s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422953418 (329h6m51.885045901s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423269190 (329h6m51.885361663s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423585622 (329h6m51.885678105s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423893889 (329h6m51.885986362s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424305283 (329h6m51.886397766s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424825754 (329h6m51.886918237s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425352837 (329h6m51.88744532s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425963780 (329h6m51.888056263s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426564014 (329h6m51.888656497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426685595 (329h6m51.888778088s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.427260630 (329h6m51.889353113s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.428079549 (329h6m51.890172032s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:07.867445156 (329h6m53.329537639s) [upload=289637]: Next returned nil segment Oct 7 06:38:07.867467089 (329h6m53.329559572s) [upload=289637]: Waiting for error group managing segments... Oct 7 06:38:08.257086538 (329h6m53.719179021s) [noise=true , address=64.49.13.138:4481 , piece_id=JO4TMBHCZZ7DQMRZQJPJFR52PBJTIZ54Q5LKATJ5ORDTWG52YKMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.336818631 (329h6m53.798911124s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.366270112 (329h6m53.828362595s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.397675292 (329h6m53.859767775s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.400627513 (329h6m53.862719996s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.414548000 (329h6m53.876640483s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.467298877 (329h6m53.92939136s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.490839973 (329h6m53.952932456s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.524333590 (329h6m53.986426073s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.534156615 (329h6m53.996249088s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:14.068606852 (329h6m59.530699325s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:17.034284555 (329h7m2.496377038s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:21.896577969 (329h7m7.358670452s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417662014 (329h7m10.879754497s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417702531 (329h7m10.879795014s) [seg_pos={0 0}, upload=289637]: Segment reached optimal threshold of 14 pieces. Oct 7 06:38:25.417898014 (329h7m10.879990497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FJGmqUskGmxYctVCEaYhRHq98m1VCVZiWL1ShEMJzmieaYo1u): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417911219 (329h7m10.880003702s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.417976213 (329h7m10.880068696s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1asDxxBjiU6yNbZ5c6uBmSWQ4QbVs5J3ErLbpKBSwqaptpNhpn): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417994398 (329h7m10.880086881s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418029004 (329h7m10.880121487s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12chhfe1weiPC1jjv8vbScP7uFVNXFr6s3UcNRkH8gsSAtGD16D): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418038491 (329h7m10.880130964s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418064501 (329h7m10.880156974s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jvSGi6s5yi55BBFJ4aErrap64wGP2WbDL9oUB6pdGiv8DM385): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418070512 (329h7m10.880162986s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418088236 (329h7m10.880180719s) [seg_pos={0 0}, upload=289637]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:38:25.418177306 (329h7m10.880269779s) [upload=289637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:38:25.418182626 (329h7m10.880275099s) [upload=289637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:38:25.730712968 (329h7m11.192805452s) [upload=289637]: Done uploading segments... Oct 7 06:38:25.730733146 (329h7m11.192825619s) [upload=289637]: Upload finished. err: <nil> Duration: 5.923302476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 8.831830279s FAIL (2025-10-07 06:41:06 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: 2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA) context deadline exceeded; piecestore: (Node ID: 12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, Piece ID: LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ) context deadline exceeded; piecestore: (Node ID: 12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, Piece ID: 7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA) context deadline exceeded; piecestore: (Node ID: 12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, Piece ID: MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q) context deadline exceeded; piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ) context deadline exceeded ======LOGS====== 38:51.525809865 (329h7m36.987902338s) [upload=289645]: Uploading segments... Oct 7 06:38:51.526041425 (329h7m36.988133908s) [upload=289645]: Got next segment. Inline: false Oct 7 06:38:51.526090008 (329h7m36.988182501s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:38:51.526096420 (329h7m36.988188893s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:38:51.786641821 (329h7m37.248734294s) [seg_pos={0 0}, upload=289645]: Begin upload segment... Oct 7 06:38:51.786704200 (329h7m37.248796673s) [seg_pos={0 0}, upload=289645]: Done begin upload segment. Oct 7 06:38:51.786748324 (329h7m37.248840807s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.786761349 (329h7m37.248853822s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787235772 (329h7m37.249328245s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787646174 (329h7m37.249738657s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.788493227 (329h7m37.25058571s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789213488 (329h7m37.251305971s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789676100 (329h7m37.251768583s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789893954 (329h7m37.251986437s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790205708 (329h7m37.252298191s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790776945 (329h7m37.252869428s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.792121125 (329h7m37.254213608s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.793644276 (329h7m37.255736759s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.795373438 (329h7m37.257465921s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796885548 (329h7m37.258978041s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796913079 (329h7m37.259005552s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.797359229 (329h7m37.259451712s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:53.629515485 (329h7m39.091607968s) [upload=289645]: Next returned nil segment Oct 7 06:38:53.629538268 (329h7m39.091630751s) [upload=289645]: Waiting for error group managing segments... Oct 7 06:38:53.918072374 (329h7m39.380164847s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.940442060 (329h7m39.402534543s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.964069099 (329h7m39.426161582s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.996913059 (329h7m39.459005542s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.999517939 (329h7m39.461610412s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:54.107690628 (329h7m39.569783112s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:14.627829239 (329h8m0.089921712s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.614758925 (329h8m1.076851408s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.670324441 (329h8m1.132416924s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.926280263 (329h8m1.388372756s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.605972466 (329h8m4.068064949s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.778892774 (329h8m4.240985257s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922160720 (329h8m4.384253203s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922200937 (329h8m4.38429343s) [seg_pos={0 0}, upload=289645]: Segment reached optimal threshold of 13 pieces. Oct 7 06:39:18.922491700 (329h8m4.384584183s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection Oct 7 06:39:18.922504335 (329h8m4.384596818s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922631556 (329h8m4.384724039s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection Oct 7 06:39:18.922650843 (329h8m4.384743317s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922662095 (329h8m4.384754578s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: context canceled; context canceled Oct 7 06:39:18.922670070 (329h8m4.384762543s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922688044 (329h8m4.384780527s) [seg_pos={0 0}, upload=289645]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:39:18.922800388 (329h8m4.384892861s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:39:18.922805427 (329h8m4.3848979s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:39:19.252191662 (329h8m4.714284146s) [upload=289645]: Done uploading segments... Oct 7 06:39:19.252213765 (329h8m4.714306238s) [upload=289645]: Upload finished. err: <nil> Duration: 8.831873091s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.380260779s FAIL (2025-10-07 06:40:37 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: 2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA) context deadline exceeded; piecestore: (Node ID: 12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, Piece ID: LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ) context deadline exceeded; piecestore: (Node ID: 12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, Piece ID: 7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA) context deadline exceeded; piecestore: (Node ID: 12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, Piece ID: MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q) context deadline exceeded; piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ) context deadline exceeded ======LOGS====== 38:51.525809865 (329h7m36.987902338s) [upload=289645]: Uploading segments... Oct 7 06:38:51.526041425 (329h7m36.988133908s) [upload=289645]: Got next segment. Inline: false Oct 7 06:38:51.526090008 (329h7m36.988182501s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:38:51.526096420 (329h7m36.988188893s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:38:51.786641821 (329h7m37.248734294s) [seg_pos={0 0}, upload=289645]: Begin upload segment... Oct 7 06:38:51.786704200 (329h7m37.248796673s) [seg_pos={0 0}, upload=289645]: Done begin upload segment. Oct 7 06:38:51.786748324 (329h7m37.248840807s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.786761349 (329h7m37.248853822s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787235772 (329h7m37.249328245s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787646174 (329h7m37.249738657s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.788493227 (329h7m37.25058571s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789213488 (329h7m37.251305971s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789676100 (329h7m37.251768583s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789893954 (329h7m37.251986437s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790205708 (329h7m37.252298191s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790776945 (329h7m37.252869428s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.792121125 (329h7m37.254213608s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.793644276 (329h7m37.255736759s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.795373438 (329h7m37.257465921s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796885548 (329h7m37.258978041s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796913079 (329h7m37.259005552s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.797359229 (329h7m37.259451712s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:53.629515485 (329h7m39.091607968s) [upload=289645]: Next returned nil segment Oct 7 06:38:53.629538268 (329h7m39.091630751s) [upload=289645]: Waiting for error group managing segments... Oct 7 06:38:53.918072374 (329h7m39.380164847s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.940442060 (329h7m39.402534543s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.964069099 (329h7m39.426161582s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.996913059 (329h7m39.459005542s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.999517939 (329h7m39.461610412s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:54.107690628 (329h7m39.569783112s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:14.627829239 (329h8m0.089921712s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.614758925 (329h8m1.076851408s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.670324441 (329h8m1.132416924s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.926280263 (329h8m1.388372756s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.605972466 (329h8m4.068064949s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.778892774 (329h8m4.240985257s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922160720 (329h8m4.384253203s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922200937 (329h8m4.38429343s) [seg_pos={0 0}, upload=289645]: Segment reached optimal threshold of 13 pieces. Oct 7 06:39:18.922491700 (329h8m4.384584183s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection Oct 7 06:39:18.922504335 (329h8m4.384596818s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922631556 (329h8m4.384724039s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection Oct 7 06:39:18.922650843 (329h8m4.384743317s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922662095 (329h8m4.384754578s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: context canceled; context canceled Oct 7 06:39:18.922670070 (329h8m4.384762543s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922688044 (329h8m4.384780527s) [seg_pos={0 0}, upload=289645]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:39:18.922800388 (329h8m4.384892861s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:39:18.922805427 (329h8m4.3848979s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:39:19.252191662 (329h8m4.714284146s) [upload=289645]: Done uploading segments... Oct 7 06:39:19.252213765 (329h8m4.714306238s) [upload=289645]: Upload finished. err: <nil> Duration: 6.380297911s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.140422114s FAIL (2025-10-07 06:40:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, Piece ID: 3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA) context deadline exceeded; piecestore: (Node ID: 123r9xrR18b29rsRY6APtib52WagU7ttLA7MHY4EQ7inh6BjdCB, Piece ID: HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ) context deadline exceeded; piecestore: (Node ID: 12wXqePK1kK7iBo6wHcbqkRmkED8LGJ3rzQNJAWa14cHYhx1oyB, Piece ID: NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A) context deadline exceeded; piecestore: (Node ID: 12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg, Piece ID: NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A) context deadline exceeded ======LOGS====== pload=289637]: Uploading piece... Oct 7 06:38:06.420968208 (329h6m51.883060691s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.421700603 (329h6m51.883793076s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422059165 (329h6m51.884151638s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422378814 (329h6m51.884471297s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422666322 (329h6m51.884758795s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422953418 (329h6m51.885045901s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423269190 (329h6m51.885361663s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423585622 (329h6m51.885678105s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423893889 (329h6m51.885986362s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424305283 (329h6m51.886397766s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424825754 (329h6m51.886918237s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425352837 (329h6m51.88744532s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425963780 (329h6m51.888056263s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426564014 (329h6m51.888656497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426685595 (329h6m51.888778088s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.427260630 (329h6m51.889353113s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.428079549 (329h6m51.890172032s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:07.867445156 (329h6m53.329537639s) [upload=289637]: Next returned nil segment Oct 7 06:38:07.867467089 (329h6m53.329559572s) [upload=289637]: Waiting for error group managing segments... Oct 7 06:38:08.257086538 (329h6m53.719179021s) [noise=true , address=64.49.13.138:4481 , piece_id=JO4TMBHCZZ7DQMRZQJPJFR52PBJTIZ54Q5LKATJ5ORDTWG52YKMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.336818631 (329h6m53.798911124s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.366270112 (329h6m53.828362595s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.397675292 (329h6m53.859767775s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.400627513 (329h6m53.862719996s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.414548000 (329h6m53.876640483s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.467298877 (329h6m53.92939136s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.490839973 (329h6m53.952932456s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.524333590 (329h6m53.986426073s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.534156615 (329h6m53.996249088s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:14.068606852 (329h6m59.530699325s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:17.034284555 (329h7m2.496377038s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:21.896577969 (329h7m7.358670452s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417662014 (329h7m10.879754497s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417702531 (329h7m10.879795014s) [seg_pos={0 0}, upload=289637]: Segment reached optimal threshold of 14 pieces. Oct 7 06:38:25.417898014 (329h7m10.879990497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FJGmqUskGmxYctVCEaYhRHq98m1VCVZiWL1ShEMJzmieaYo1u): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417911219 (329h7m10.880003702s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.417976213 (329h7m10.880068696s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1asDxxBjiU6yNbZ5c6uBmSWQ4QbVs5J3ErLbpKBSwqaptpNhpn): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417994398 (329h7m10.880086881s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418029004 (329h7m10.880121487s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12chhfe1weiPC1jjv8vbScP7uFVNXFr6s3UcNRkH8gsSAtGD16D): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418038491 (329h7m10.880130964s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418064501 (329h7m10.880156974s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jvSGi6s5yi55BBFJ4aErrap64wGP2WbDL9oUB6pdGiv8DM385): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418070512 (329h7m10.880162986s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418088236 (329h7m10.880180719s) [seg_pos={0 0}, upload=289637]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:38:25.418177306 (329h7m10.880269779s) [upload=289637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:38:25.418182626 (329h7m10.880275099s) [upload=289637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:38:25.730712968 (329h7m11.192805452s) [upload=289637]: Done uploading segments... Oct 7 06:38:25.730733146 (329h7m11.192825619s) [upload=289637]: Upload finished. err: <nil> Duration: 6.140455758s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.143802666s FAIL (2025-10-07 06:39:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: 2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA) context deadline exceeded; piecestore: (Node ID: 12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, Piece ID: LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ) context deadline exceeded; piecestore: (Node ID: 12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, Piece ID: 7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA) context deadline exceeded; piecestore: (Node ID: 12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, Piece ID: MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q) context deadline exceeded; piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ) context deadline exceeded ======LOGS====== 38:51.525809865 (329h7m36.987902338s) [upload=289645]: Uploading segments... Oct 7 06:38:51.526041425 (329h7m36.988133908s) [upload=289645]: Got next segment. Inline: false Oct 7 06:38:51.526090008 (329h7m36.988182501s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:38:51.526096420 (329h7m36.988188893s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:38:51.786641821 (329h7m37.248734294s) [seg_pos={0 0}, upload=289645]: Begin upload segment... Oct 7 06:38:51.786704200 (329h7m37.248796673s) [seg_pos={0 0}, upload=289645]: Done begin upload segment. Oct 7 06:38:51.786748324 (329h7m37.248840807s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.786761349 (329h7m37.248853822s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787235772 (329h7m37.249328245s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787646174 (329h7m37.249738657s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.788493227 (329h7m37.25058571s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789213488 (329h7m37.251305971s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789676100 (329h7m37.251768583s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789893954 (329h7m37.251986437s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790205708 (329h7m37.252298191s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790776945 (329h7m37.252869428s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.792121125 (329h7m37.254213608s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.793644276 (329h7m37.255736759s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.795373438 (329h7m37.257465921s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796885548 (329h7m37.258978041s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796913079 (329h7m37.259005552s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.797359229 (329h7m37.259451712s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:53.629515485 (329h7m39.091607968s) [upload=289645]: Next returned nil segment Oct 7 06:38:53.629538268 (329h7m39.091630751s) [upload=289645]: Waiting for error group managing segments... Oct 7 06:38:53.918072374 (329h7m39.380164847s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.940442060 (329h7m39.402534543s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.964069099 (329h7m39.426161582s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.996913059 (329h7m39.459005542s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.999517939 (329h7m39.461610412s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:54.107690628 (329h7m39.569783112s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:14.627829239 (329h8m0.089921712s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.614758925 (329h8m1.076851408s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.670324441 (329h8m1.132416924s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.926280263 (329h8m1.388372756s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.605972466 (329h8m4.068064949s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.778892774 (329h8m4.240985257s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922160720 (329h8m4.384253203s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922200937 (329h8m4.38429343s) [seg_pos={0 0}, upload=289645]: Segment reached optimal threshold of 13 pieces. Oct 7 06:39:18.922491700 (329h8m4.384584183s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection Oct 7 06:39:18.922504335 (329h8m4.384596818s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922631556 (329h8m4.384724039s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection Oct 7 06:39:18.922650843 (329h8m4.384743317s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922662095 (329h8m4.384754578s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: context canceled; context canceled Oct 7 06:39:18.922670070 (329h8m4.384762543s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922688044 (329h8m4.384780527s) [seg_pos={0 0}, upload=289645]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:39:18.922800388 (329h8m4.384892861s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:39:18.922805427 (329h8m4.3848979s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:39:19.252191662 (329h8m4.714284146s) [upload=289645]: Done uploading segments... Oct 7 06:39:19.252213765 (329h8m4.714306238s) [upload=289645]: Upload finished. err: <nil> Duration: 6.143826311s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 16.412670489s FAIL (2025-10-07 06:39:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, Piece ID: 3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA) context deadline exceeded; piecestore: (Node ID: 123r9xrR18b29rsRY6APtib52WagU7ttLA7MHY4EQ7inh6BjdCB, Piece ID: HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ) context deadline exceeded; piecestore: (Node ID: 12wXqePK1kK7iBo6wHcbqkRmkED8LGJ3rzQNJAWa14cHYhx1oyB, Piece ID: NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A) context deadline exceeded; piecestore: (Node ID: 12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg, Piece ID: NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A) context deadline exceeded ======LOGS====== pload=289637]: Uploading piece... Oct 7 06:38:06.420968208 (329h6m51.883060691s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.421700603 (329h6m51.883793076s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422059165 (329h6m51.884151638s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422378814 (329h6m51.884471297s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422666322 (329h6m51.884758795s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422953418 (329h6m51.885045901s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423269190 (329h6m51.885361663s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423585622 (329h6m51.885678105s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423893889 (329h6m51.885986362s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424305283 (329h6m51.886397766s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424825754 (329h6m51.886918237s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425352837 (329h6m51.88744532s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425963780 (329h6m51.888056263s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426564014 (329h6m51.888656497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426685595 (329h6m51.888778088s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.427260630 (329h6m51.889353113s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.428079549 (329h6m51.890172032s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:07.867445156 (329h6m53.329537639s) [upload=289637]: Next returned nil segment Oct 7 06:38:07.867467089 (329h6m53.329559572s) [upload=289637]: Waiting for error group managing segments... Oct 7 06:38:08.257086538 (329h6m53.719179021s) [noise=true , address=64.49.13.138:4481 , piece_id=JO4TMBHCZZ7DQMRZQJPJFR52PBJTIZ54Q5LKATJ5ORDTWG52YKMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.336818631 (329h6m53.798911124s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.366270112 (329h6m53.828362595s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.397675292 (329h6m53.859767775s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.400627513 (329h6m53.862719996s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.414548000 (329h6m53.876640483s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.467298877 (329h6m53.92939136s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.490839973 (329h6m53.952932456s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.524333590 (329h6m53.986426073s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.534156615 (329h6m53.996249088s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:14.068606852 (329h6m59.530699325s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:17.034284555 (329h7m2.496377038s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:21.896577969 (329h7m7.358670452s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417662014 (329h7m10.879754497s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417702531 (329h7m10.879795014s) [seg_pos={0 0}, upload=289637]: Segment reached optimal threshold of 14 pieces. Oct 7 06:38:25.417898014 (329h7m10.879990497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FJGmqUskGmxYctVCEaYhRHq98m1VCVZiWL1ShEMJzmieaYo1u): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417911219 (329h7m10.880003702s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.417976213 (329h7m10.880068696s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1asDxxBjiU6yNbZ5c6uBmSWQ4QbVs5J3ErLbpKBSwqaptpNhpn): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417994398 (329h7m10.880086881s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418029004 (329h7m10.880121487s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12chhfe1weiPC1jjv8vbScP7uFVNXFr6s3UcNRkH8gsSAtGD16D): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418038491 (329h7m10.880130964s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418064501 (329h7m10.880156974s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jvSGi6s5yi55BBFJ4aErrap64wGP2WbDL9oUB6pdGiv8DM385): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418070512 (329h7m10.880162986s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418088236 (329h7m10.880180719s) [seg_pos={0 0}, upload=289637]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:38:25.418177306 (329h7m10.880269779s) [upload=289637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:38:25.418182626 (329h7m10.880275099s) [upload=289637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:38:25.730712968 (329h7m11.192805452s) [upload=289637]: Done uploading segments... Oct 7 06:38:25.730733146 (329h7m11.192825619s) [upload=289637]: Upload finished. err: <nil> Duration: 16.412699355s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.003229017s FAIL (2025-10-07 06:39:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1EZTRkB6M2F4mwRbYSvsv3ybGk1Gz225pDd18RNBk1xe1SFdGA, Piece ID: 2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA) context deadline exceeded; piecestore: (Node ID: 12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, Piece ID: LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q) context deadline exceeded; piecestore: (Node ID: 1mvEBa8qdXMR14Gcz96iRQUKyscWCUDnFXh98NQJC3ndoWkkq3, Piece ID: OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ) context deadline exceeded; piecestore: (Node ID: 1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz, Piece ID: YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ) context deadline exceeded; piecestore: (Node ID: 12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, Piece ID: 7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA) context deadline exceeded; piecestore: (Node ID: 12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, Piece ID: MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q) context deadline exceeded; piecestore: (Node ID: 1srnb2e1ZCtNSpkLsFuMNR6BMdYzH3tqfxZii3qwBnRhbXcee9, Piece ID: AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A) context deadline exceeded; piecestore: (Node ID: 1XATjg9fmRUK8ianyYsEgDMvA3wByoKPdztMnM98XSdYBxyFdr, Piece ID: SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ) context deadline exceeded ======LOGS====== 38:51.525809865 (329h7m36.987902338s) [upload=289645]: Uploading segments... Oct 7 06:38:51.526041425 (329h7m36.988133908s) [upload=289645]: Got next segment. Inline: false Oct 7 06:38:51.526090008 (329h7m36.988182501s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 06:38:51.526096420 (329h7m36.988188893s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 06:38:51.786641821 (329h7m37.248734294s) [seg_pos={0 0}, upload=289645]: Begin upload segment... Oct 7 06:38:51.786704200 (329h7m37.248796673s) [seg_pos={0 0}, upload=289645]: Done begin upload segment. Oct 7 06:38:51.786748324 (329h7m37.248840807s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.786761349 (329h7m37.248853822s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787235772 (329h7m37.249328245s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.787646174 (329h7m37.249738657s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.788493227 (329h7m37.25058571s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789213488 (329h7m37.251305971s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789676100 (329h7m37.251768583s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.789893954 (329h7m37.251986437s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790205708 (329h7m37.252298191s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.790776945 (329h7m37.252869428s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.792121125 (329h7m37.254213608s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.793644276 (329h7m37.255736759s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.795373438 (329h7m37.257465921s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796885548 (329h7m37.258978041s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.796913079 (329h7m37.259005552s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:51.797359229 (329h7m37.259451712s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Uploading piece... Oct 7 06:38:53.629515485 (329h7m39.091607968s) [upload=289645]: Next returned nil segment Oct 7 06:38:53.629538268 (329h7m39.091630751s) [upload=289645]: Waiting for error group managing segments... Oct 7 06:38:53.918072374 (329h7m39.380164847s) [noise=true , address=45.91.116.10:4083 , piece_id=HUMQHWQUY3NYSRVZSKO3U27CAVLPK2ZGO2D4CKFJP7DK5YUC6RGQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.940442060 (329h7m39.402534543s) [noise=true , address=45.91.116.13:4154 , piece_id=EN5T4HK3BYZ5W4SUXSMCEUELDBYKFJ3PAWSC52ZNGNSNAYOIDDRQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.964069099 (329h7m39.426161582s) [noise=true , address=77.90.31.11:4553 , piece_id=YBNFOXW3YRF2IKNDQXXFSQRYATIDZJSMB5J5C5EG33SQOAIOFPTQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.996913059 (329h7m39.459005542s) [noise=true , address=77.90.31.10:4518 , piece_id=OLDIO3LC26CBWQFJSYJGACKC5QQYP6HBIFOLAMEE2X46GXILVJQQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:53.999517939 (329h7m39.461610412s) [noise=true , address=77.90.31.16:4734 , piece_id=AGDHQT2PEYJHFQQEPDYFKPUCZGEBWTARCMND6E6PO6GF3N6VGS7A, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:38:54.107690628 (329h7m39.569783112s) [noise=true , address=77.90.31.19:4841 , piece_id=2VIJTTTODTPTVEXSVIM6KKC55NFQ2YX6373LJ7HLO7XAF76BUCZA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:14.627829239 (329h8m0.089921712s) [noise=true , address=145.79.227.13:4801 , piece_id=MVE7OTIQYBPZIGGMN7U5WLZ3ADKJO2WQJOV2VJK4FSVT4FWSA74Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.614758925 (329h8m1.076851408s) [noise=true , address=145.79.227.14:4840 , piece_id=LQOUC7JIDYTLYAWDBVHFTJXLSZKYNVPSJC7VA2ISKFCEP7AVW77Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.670324441 (329h8m1.132416924s) [noise=true , address=145.79.227.6:4258 , piece_id=J6QVE6CKRG4DYDZCTWEBDAIARFIG54XZVH6EDLMZRJVNSG2SLDDQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:15.926280263 (329h8m1.388372756s) [noise=true , address=145.79.227.11:4729 , piece_id=MJXILD337KBOJQSUAJUEIGM7QDWCLPDZ2R2AHL4RF5KXMKNUIAOA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.605972466 (329h8m4.068064949s) [noise=true , address=145.79.227.17:4971 , piece_id=7CC35PAQKEVDEW6T3GYDBXNDFLCW26QQMZOLZ2PKANOOTROBRUWA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.778892774 (329h8m4.240985257s) [noise=true , address=145.79.227.19:4078 , piece_id=SO7ZX2XZFNV4UNWZBAQ6IKL23QVOJUDZTEBIVP7JRO4IGIS6XFKQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922160720 (329h8m4.384253203s) [noise=true , address=145.79.227.5:4221 , piece_id=LC4WMGO3PFIJ53JTGSBMVIU4H23EDUXVRBCJZRPQJ4YFVNK56JRA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: <nil> Oct 7 06:39:18.922200937 (329h8m4.38429343s) [seg_pos={0 0}, upload=289645]: Segment reached optimal threshold of 13 pieces. Oct 7 06:39:18.922491700 (329h8m4.384584183s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV): protocol: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46156->145.79.227.14:4837: use of closed network connection Oct 7 06:39:18.922504335 (329h8m4.384596818s) [noise=true , address=145.79.227.14:4837 , piece_id=UWLVGI5SKIB4VHQV6PNF5ISEW7FEH6V7BE75YDJAD7NS62KSDS4Q, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922631556 (329h8m4.384724039s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK): protocol: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34338->145.79.227.3:4150: use of closed network connection Oct 7 06:39:18.922650843 (329h8m4.384743317s) [noise=true , address=145.79.227.3:4150 , piece_id=YN5Q5CFE26ST4E7HWWDNWB5DJZZEA5RTVLDVVQOHNBR7DWGE3QJA, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922662095 (329h8m4.384754578s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE): protocol: context canceled; context canceled Oct 7 06:39:18.922670070 (329h8m4.384762543s) [noise=true , address=145.79.227.7:4293 , piece_id=P6INSWYPMPTEXK4ZJLRE7PGHZ5U4QS6IKYCWQ6FTNFSOT5BKPKCQ, seg_pos={0 0}, upload=289645]: Long tail cancellation due to optimal threshold Oct 7 06:39:18.922688044 (329h8m4.384780527s) [seg_pos={0 0}, upload=289645]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 06:39:18.922800388 (329h8m4.384892861s) [upload=289645]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:39:18.922805427 (329h8m4.3848979s) [upload=289645]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:39:19.252191662 (329h8m4.714284146s) [upload=289645]: Done uploading segments... Oct 7 06:39:19.252213765 (329h8m4.714306238s) [upload=289645]: Upload finished. err: <nil> Duration: 30.003253395s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.003510048s FAIL (2025-10-07 06:39:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 121EfsJiXbm2HktqinYqjYaHWRyGiu2wA9fq4PQXpyizYttDCi1, Piece ID: 3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA) context deadline exceeded; piecestore: (Node ID: 123r9xrR18b29rsRY6APtib52WagU7ttLA7MHY4EQ7inh6BjdCB, Piece ID: HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ) context deadline exceeded; piecestore: (Node ID: 12wXqePK1kK7iBo6wHcbqkRmkED8LGJ3rzQNJAWa14cHYhx1oyB, Piece ID: NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A) context deadline exceeded; piecestore: (Node ID: 12UtzGEdmMaJAFkfprwLpTuxP2XDFtiN3Tgcqpnf9UtHVQzdhEg, Piece ID: NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A) context deadline exceeded ======LOGS====== pload=289637]: Uploading piece... Oct 7 06:38:06.420968208 (329h6m51.883060691s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.421700603 (329h6m51.883793076s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422059165 (329h6m51.884151638s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422378814 (329h6m51.884471297s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422666322 (329h6m51.884758795s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.422953418 (329h6m51.885045901s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423269190 (329h6m51.885361663s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423585622 (329h6m51.885678105s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.423893889 (329h6m51.885986362s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424305283 (329h6m51.886397766s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.424825754 (329h6m51.886918237s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425352837 (329h6m51.88744532s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.425963780 (329h6m51.888056263s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426564014 (329h6m51.888656497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.426685595 (329h6m51.888778088s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.427260630 (329h6m51.889353113s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:06.428079549 (329h6m51.890172032s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Uploading piece... Oct 7 06:38:07.867445156 (329h6m53.329537639s) [upload=289637]: Next returned nil segment Oct 7 06:38:07.867467089 (329h6m53.329559572s) [upload=289637]: Waiting for error group managing segments... Oct 7 06:38:08.257086538 (329h6m53.719179021s) [noise=true , address=64.49.13.138:4481 , piece_id=JO4TMBHCZZ7DQMRZQJPJFR52PBJTIZ54Q5LKATJ5ORDTWG52YKMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.336818631 (329h6m53.798911124s) [noise=true , address=77.90.31.16:4752 , piece_id=GGMQ4RKEZO4AOKQESSBVZGCV2PAESIP74OXWDYRR67STPGFLIGMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.366270112 (329h6m53.828362595s) [noise=true , address=45.91.116.7:4035 , piece_id=NLN5E3QFUG65D5UHUJF5YX4QKIZOD2QWZLGDH7J6LYFOI2DMT44A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.397675292 (329h6m53.859767775s) [noise=true , address=45.91.116.5:4305 , piece_id=3JS4DSBELVB2CPMTUYM2KIF3R6V6Z6TMZWDEGYHMSEMPBFFZACWA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.400627513 (329h6m53.862719996s) [noise=true , address=77.90.31.14:4689 , piece_id=CNBDC3G4AXWUTJLAUE6H3HKSZWDHSAOBV6NNQWJASJAY5ZGCSTCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.414548000 (329h6m53.876640483s) [noise=true , address=45.91.116.14:4202 , piece_id=NWVFKVJJG2FBBTWKNEPCP74AGSRXLH76M36Z3FMHRFZVMJN65H5A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.467298877 (329h6m53.92939136s) [noise=true , address=45.91.116.12:4243 , piece_id=GMJJRZBK4HQU6P3F262NCNZMGVFDPI4L2Y2RE7NG2LVAJGANTW4Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.490839973 (329h6m53.952932456s) [noise=true , address=77.90.31.13:4636 , piece_id=HCCUJGE6LVENYYTHLM6GSSAZA37B67SO6J7FV6YKAFWD73YK6FUQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.524333590 (329h6m53.986426073s) [noise=true , address=77.90.31.10:4531 , piece_id=YQHPDTISMC4NNADWQHOOTAX7MFK2AVS6YBZ5N7FITWXOAKIUL4GA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:08.534156615 (329h6m53.996249088s) [noise=true , address=77.90.31.15:4726 , piece_id=I6Q4OXGDBRR2JFTNMJE2YVDZCRB6W5C6CO3DBJUYYZRZ7LCMW3PA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:14.068606852 (329h6m59.530699325s) [noise=true , address=145.79.227.16:4922 , piece_id=2O3K54TXABJAIY4WIAPGW6KP2EQP76YPM5ZK4UQ74IEMGBLGZRTQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:17.034284555 (329h7m2.496377038s) [noise=true , address=145.79.227.28:4750 , piece_id=GTSGPDEYA6JH2QP52Z23NG7PZIEVO6SCYAGNBEV6EDU7KCF4TO3Q, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:21.896577969 (329h7m7.358670452s) [noise=true , address=145.79.227.15:4902 , piece_id=DE6VKF5DZMHB7FMD4ISMMOXHPQ7ELNWYKN3AEEFHP6JRA3BHSS7A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417662014 (329h7m10.879754497s) [noise=true , address=145.79.227.33:4881 , piece_id=TVCP2HX25TKXSRE6W56ZRAPAZOP6OMOBDULNQRACRCUKZJTBWG4A, seg_pos={0 0}, upload=289637]: Done uploading piece. err: <nil> Oct 7 06:38:25.417702531 (329h7m10.879795014s) [seg_pos={0 0}, upload=289637]: Segment reached optimal threshold of 14 pieces. Oct 7 06:38:25.417898014 (329h7m10.879990497s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1FJGmqUskGmxYctVCEaYhRHq98m1VCVZiWL1ShEMJzmieaYo1u): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417911219 (329h7m10.880003702s) [noise=true , address=145.79.227.36:4546 , piece_id=3ORNY7JOVZKAQUUICWZTMGBF3GWNGDPLEFQEZYEJEUEE67KCSIMA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.417976213 (329h7m10.880068696s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1asDxxBjiU6yNbZ5c6uBmSWQ4QbVs5J3ErLbpKBSwqaptpNhpn): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.417994398 (329h7m10.880086881s) [noise=true , address=145.79.227.30:4793 , piece_id=POXMMUFZX2BEEAW4N4KSNJES64VEF7AFS53DOSS7NLWDPJXV6DPQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418029004 (329h7m10.880121487s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12chhfe1weiPC1jjv8vbScP7uFVNXFr6s3UcNRkH8gsSAtGD16D): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418038491 (329h7m10.880130964s) [noise=true , address=145.79.227.38:4000 , piece_id=NVPMBDVV22HTSD6M7I27DWQZVBLTCBWC4WJHLYWJUO2IH5OPXYCA, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418064501 (329h7m10.880156974s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12jvSGi6s5yi55BBFJ4aErrap64wGP2WbDL9oUB6pdGiv8DM385): protocol: expected piece hash; context canceled; EOF Oct 7 06:38:25.418070512 (329h7m10.880162986s) [noise=true , address=145.79.227.21:4486 , piece_id=OGEBMFXSMLGVUQWBRGKNL2N2YVM7QFQYOIDNPXXKVJP2TSG3UWHQ, seg_pos={0 0}, upload=289637]: Long tail cancellation due to optimal threshold Oct 7 06:38:25.418088236 (329h7m10.880180719s) [seg_pos={0 0}, upload=289637]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 06:38:25.418177306 (329h7m10.880269779s) [upload=289637]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 06:38:25.418182626 (329h7m10.880275099s) [upload=289637]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 06:38:25.730712968 (329h7m11.192805452s) [upload=289637]: Done uploading segments... Oct 7 06:38:25.730733146 (329h7m11.192825619s) [upload=289637]: Upload finished. err: <nil> Duration: 1m0.00353749s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.030631805s FAIL (2025-10-07 06:29:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.030680738s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.016165452s FAIL (2025-10-07 06:28:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.016199105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.203138417s FAIL (2025-10-07 06:27:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.203267141s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.57455356s FAIL (2025-10-07 06:27:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.574588947s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 6.136882816s FAIL (2025-10-07 06:26:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.136915589s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XX...X......................................................] duration: 6.212818258s FAIL (2025-10-07 06:25:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.212853135s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XX...X.....................................................] duration: 6.185859276s FAIL (2025-10-07 06:24:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.185905824s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [X.XX...X....................................................] duration: 6.329859281s FAIL (2025-10-07 06:24:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.329906513s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.X.XX...X...................................................] duration: 6.492361786s FAIL (2025-10-07 06:23:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.492393697s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..X.XX...X..................................................] duration: 7.236267273s FAIL (2025-10-07 06:23:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.236418612s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...X.XX...X.................................................] duration: 6.410444886s FAIL (2025-10-07 06:22:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.410638144s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....X.XX...X................................................] duration: 7.443897873s FAIL (2025-10-07 06:22:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.443929875s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....X.XX...X...............................................] duration: 6.185698447s FAIL (2025-10-07 06:21:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.185728856s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......X.XX...X..............................................] duration: 6.964258565s FAIL (2025-10-07 06:21:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.964349557s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......X.XX...X.............................................] duration: 7.50110635s FAIL (2025-10-07 06:20:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.501142418s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........X.XX...X............................................] duration: 7.300542252s FAIL (2025-10-07 06:19:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.300563061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........X.XX...X...........................................] duration: 7.316117531s FAIL (2025-10-07 06:18:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.316190491s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........X.XX...X..........................................] duration: 8.661062714s FAIL (2025-10-07 06:17:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.661090987s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........X.XX...X.........................................] duration: 8.533217439s FAIL (2025-10-07 06:16:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.533250562s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............X.XX...X........................................] duration: 15.04442568s FAIL (2025-10-07 06:16:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 15.044459975s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............X.XX...X.......................................] duration: 8.829013596s FAIL (2025-10-07 06:15:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.829058483s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............X.XX...X......................................] duration: 9.058493242s FAIL (2025-10-07 06:15:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 9.058528329s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............X.XX...X.....................................] duration: 7.713400539s FAIL (2025-10-07 06:14:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.71343297s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................X.XX...X....................................] duration: 7.133285951s FAIL (2025-10-07 06:13:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.133319364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................X.XX...X...................................] duration: 8.914508309s FAIL (2025-10-07 06:12:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.914542323s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)