Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................X.XX...X...........................] duration: 5.999536552s FAIL (2025-10-07 06:05:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.999566469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................X.XX...X..........................] duration: 6.777781741s FAIL (2025-10-07 06:05:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.777816898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................X.XX...X.........................] duration: 6.15602399s FAIL (2025-10-07 06:04:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.156055912s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................X.XX...X........................] duration: 6.443560456s FAIL (2025-10-07 06:03:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.443597998s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................X.XX...X.......................] duration: 6.39472892s FAIL (2025-10-07 06:03:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.394763616s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................X.XX...X......................] duration: 6.578220643s FAIL (2025-10-07 06:02:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.578246423s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................X.XX...X.....................] duration: 6.369743811s FAIL (2025-10-07 06:01:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.369774661s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................X.XX...X....................] duration: 6.852401317s FAIL (2025-10-07 06:00:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.8524401s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................X.XX...X...................] duration: 7.552566358s FAIL (2025-10-07 06:00:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.552627505s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................X.XX...X..................] duration: 7.386204694s FAIL (2025-10-07 05:59:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.386238721s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................X.XX...X.................] duration: 8.705006141s FAIL (2025-10-07 05:59:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.705046057s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................X.XX...X................] duration: 7.395901187s FAIL (2025-10-07 05:58:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.395932616s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................X.XX...X...............] duration: 6.326032614s FAIL (2025-10-07 05:57:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.326069454s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................X.XX...X..............] duration: 6.411918401s FAIL (2025-10-07 05:56:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.411954639s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................X.XX...X.............] duration: 6.645550728s FAIL (2025-10-07 05:55:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.645584191s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................X.XX...X............] duration: 6.904195356s FAIL (2025-10-07 05:55:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.904242686s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................................X.XX...X...........] duration: 6.219302354s FAIL (2025-10-07 05:54:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.219340708s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................................X.XX...X..........] duration: 6.275868411s FAIL (2025-10-07 05:53:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.275900872s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................................X.XX...X.........] duration: 6.184457787s FAIL (2025-10-07 05:52:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.184480421s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................................X.XX...X........] duration: 6.232135577s FAIL (2025-10-07 05:52:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.232164232s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................X.XX...X.......] duration: 8.74969446s FAIL (2025-10-07 05:51:31 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.749723565s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................X.XX...X......] duration: 8.69286345s FAIL (2025-10-07 05:51:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.692896203s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................X.XX...X.....] duration: 8.867163318s FAIL (2025-10-07 05:50:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.867198204s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................X.XX...X....] duration: 8.538991183s FAIL (2025-10-07 05:49:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.539053361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................X.XX...X...] duration: 7.755942921s FAIL (2025-10-07 05:49:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.755975031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................X.XX...X..] duration: 7.739244006s FAIL (2025-10-07 05:48:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.73927304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................X.XX...X.] duration: 7.746859299s FAIL (2025-10-07 05:48:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.746889405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................X.XX...X] duration: 30.291759802s FAIL (2025-10-07 05:47:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 05:47:24.320708065 (328h16m9.782800538s) [upload=288869]: Starting upload Oct 7 05:47:24.320936591 (328h16m9.783029074s) [upload=288869]: Done starting upload Oct 7 05:47:24.321067961 (328h16m9.783160504s) [upload=288869]: Uploading segments... Oct 7 05:47:24.321178582 (328h16m9.783271055s) [upload=288869]: Got next segment. Inline: false Oct 7 05:47:24.321232023 (328h16m9.783324496s) [upload=288869]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:47:24.321237324 (328h16m9.783329807s) [upload=288869]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:47:24.587620061 (328h16m10.049712544s) [seg_pos={0 0}, upload=288869]: Begin upload segment... Oct 7 05:47:24.587689874 (328h16m10.049782347s) [seg_pos={0 0}, upload=288869]: Done begin upload segment. Oct 7 05:47:24.587724390 (328h16m10.049816873s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588225804 (328h16m10.050318277s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.588970011 (328h16m10.051062494s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589420819 (328h16m10.051513293s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.589841450 (328h16m10.051933923s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590174615 (328h16m10.052267088s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590561061 (328h16m10.052653534s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.590939261 (328h16m10.053031734s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592284082 (328h16m10.054376575s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.592800124 (328h16m10.054892607s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.594312444 (328h16m10.056404928s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.596232410 (328h16m10.058324883s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.598113622 (328h16m10.060206095s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.599687219 (328h16m10.061779702s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.601737883 (328h16m10.063830357s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:24.603003002 (328h16m10.065095475s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Uploading piece... Oct 7 05:47:26.257346785 (328h16m11.719439269s) [upload=288869]: Next returned nil segment Oct 7 05:47:26.257368967 (328h16m11.71946145s) [upload=288869]: Waiting for error group managing segments... Oct 7 05:47:26.646826488 (328h16m12.108918971s) [noise=true , address=45.91.116.12:4227 , piece_id=B3FC2GECL23VCHBSXQZON66GFYJ4KVZJFRH5OX65AUZX6XO7NNYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.652249685 (328h16m12.114342158s) [noise=true , address=45.91.116.5:4299 , piece_id=RJ7ZTCSKOCMDB5B4J7PIAO3NC52CGDVVRRQO4WKS2RUI4TCFIU6Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.720838907 (328h16m12.18293139s) [noise=true , address=77.90.31.15:4699 , piece_id=FWC5JHYSTZIQPIN4DS4P3YPTWNDBVFHVVWUOVKPUAUZPOG7U4JFQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.750520836 (328h16m12.212613319s) [noise=true , address=45.91.116.4:4937 , piece_id=T4NFXCSR6KN4ZKM45TBGMQMQPIY57XP6M42ZOD7IL6TFQ3UOFAMQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.763888880 (328h16m12.225981353s) [noise=true , address=77.90.31.12:4592 , piece_id=SLADRIXAAOBX2TJPETQV4O6UEX4PI5ZQCOG4YJJ3DEVTJ2EPSU5A, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.789520176 (328h16m12.251612659s) [noise=true , address=45.91.116.6:4973 , piece_id=CATB6DKLR2GMYTPCXQJINS4GFN4HUP6IIKWRKOU3YMC64G5VLXYQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.796814966 (328h16m12.258907449s) [noise=true , address=45.91.116.6:4975 , piece_id=IEFEHFARZQMNT6SV5OHFULIFVDABEMH2CHJH6VPVJZRQ4Q7TJ7GQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:26.838614818 (328h16m12.300707311s) [noise=true , address=45.91.116.10:4081 , piece_id=HOSLSV6AVDHGW4JXNQA5X6BVYCHYGALJ66ZYGFBRLC7DXVIZCRPQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:49.589925234 (328h16m35.052017707s) [noise=true , address=145.79.227.19:4079 , piece_id=S7GAV6D5N5VXWVV74MZCKNODSFCPL22ZYOBMFU75XOVEUSAA223Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:50.385674869 (328h16m35.847767352s) [noise=true , address=145.79.227.2:4114 , piece_id=YKVP3FT72KCBKXPCTF2XVK6TEQSEFBNBMQUUCQ2GVXOCQKJ2NGLQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: <nil> Oct 7 05:47:53.571767142 (328h16m39.033859625s) [noise=true , address=145.79.227.18:4006 , piece_id=ZFPKP6MR4WD5SDS5E6WLJAYNMUJY7UZ6JHQS327SEGMJPBS7HXIA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46402->145.79.227.18:4006: use of closed network connection Oct 7 05:47:53.571800826 (328h16m39.033893309s) [noise=true , address=145.79.227.14:4837 , piece_id=VSRKZSDVICFHNCIFJZYBZUX4ULP7LRID7KEWKG6SSVSLEOTEZRZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571870118 (328h16m39.033962591s) [noise=true , address=145.79.227.14:4840 , piece_id=3KQ3SKQ4S3M4OD55EIETD75PVH7KX5ZJODRYELSVLY6BTVDWQQ2Q, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.571999234 (328h16m39.034091728s) [noise=true , address=145.79.227.17:4972 , piece_id=BVTAGHPI57EJASOG36O3HSN7AUVM4H3GMO5F2DRJBVWBIMHXCINQ, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:51846->145.79.227.17:4972: use of closed network connection Oct 7 05:47:53.572017960 (328h16m39.034110443s) [noise=true , address=145.79.227.11:4731 , piece_id=VXGD72DCAGGENEIJDFFSIYRJEP2GNPLLEZOWGJHND5PWJY2575FA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9, address:145.79.227.11:4731): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:47:53.572085318 (328h16m39.034177801s) [noise=true , address=145.79.227.17:4971 , piece_id=CK6DFOCHYU3R3Z2ABCOG5O4XVD6KWLC23C3E34Y6OWCLMXX7CBZA, seg_pos={0 0}, upload=288869]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53714->145.79.227.17:4971: use of closed network connection Oct 7 05:47:53.572113882 (328h16m39.034206366s) [seg_pos={0 0}, upload=288869]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 05:47:53.861196946 (328h16m39.323289429s) [upload=288869]: Done uploading segments... Oct 7 05:47:53.861233696 (328h16m39.323326179s) [upload=288869]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.291785172s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.....................................................X.XX...] duration: 7.231178898s FAIL (2025-10-07 05:47:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 05:45:16.258577472 (328h14m1.720669955s) [upload=288839]: Starting upload Oct 7 05:45:16.258719863 (328h14m1.720812346s) [upload=288839]: Done starting upload Oct 7 05:45:16.259103042 (328h14m1.721195525s) [upload=288839]: Uploading segments... Oct 7 05:45:16.259160101 (328h14m1.721252584s) [upload=288839]: Got next segment. Inline: false Oct 7 05:45:16.259196861 (328h14m1.721289344s) [upload=288839]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:45:16.259235084 (328h14m1.721327567s) [upload=288839]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:45:16.530011259 (328h14m1.992103742s) [seg_pos={0 0}, upload=288839]: Begin upload segment... Oct 7 05:45:16.530086823 (328h14m1.992179306s) [seg_pos={0 0}, upload=288839]: Done begin upload segment. Oct 7 05:45:16.530116119 (328h14m1.992208602s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530174690 (328h14m1.992267174s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530558932 (328h14m1.992651415s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530891485 (328h14m1.992983968s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531332224 (328h14m1.993424707s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531784496 (328h14m1.993876979s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532132368 (328h14m1.994224851s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532453750 (328h14m1.994546233s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532899228 (328h14m1.994991701s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534192571 (328h14m1.996285054s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534707100 (328h14m1.996799583s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535198395 (328h14m1.997290868s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535673079 (328h14m1.997765562s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535944537 (328h14m1.99803702s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536358194 (328h14m1.998450677s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536696960 (328h14m1.998789443s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:18.269979062 (328h14m3.732071555s) [upload=288839]: Next returned nil segment Oct 7 05:45:18.270002597 (328h14m3.73209508s) [upload=288839]: Waiting for error group managing segments... Oct 7 05:45:18.716493386 (328h14m4.178585869s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.735914286 (328h14m4.198006769s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.746153042 (328h14m4.208245535s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.764417600 (328h14m4.226510083s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.776451344 (328h14m4.238543827s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.781866084 (328h14m4.243958567s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.790674517 (328h14m4.252767s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.794717826 (328h14m4.256810319s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.823491756 (328h14m4.285584229s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:21.804432647 (328h14m7.26652513s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:27.473695541 (328h14m12.935788024s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:41.787793147 (328h14m27.249885631s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:45.508437429 (328h14m30.970529902s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508505497 (328h14m30.97059798s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508563087 (328h14m30.97065558s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508632800 (328h14m30.970725383s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508688486 (328h14m30.97078097s) [seg_pos={0 0}, upload=288839]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 05:45:45.775703629 (328h14m31.237796112s) [upload=288839]: Done uploading segments... Oct 7 05:45:45.775732604 (328h14m31.237825087s) [upload=288839]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.231281061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [......................................................X.XX..] duration: 6.055216165s FAIL (2025-10-07 05:46:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 05:45:16.258577472 (328h14m1.720669955s) [upload=288839]: Starting upload Oct 7 05:45:16.258719863 (328h14m1.720812346s) [upload=288839]: Done starting upload Oct 7 05:45:16.259103042 (328h14m1.721195525s) [upload=288839]: Uploading segments... Oct 7 05:45:16.259160101 (328h14m1.721252584s) [upload=288839]: Got next segment. Inline: false Oct 7 05:45:16.259196861 (328h14m1.721289344s) [upload=288839]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:45:16.259235084 (328h14m1.721327567s) [upload=288839]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:45:16.530011259 (328h14m1.992103742s) [seg_pos={0 0}, upload=288839]: Begin upload segment... Oct 7 05:45:16.530086823 (328h14m1.992179306s) [seg_pos={0 0}, upload=288839]: Done begin upload segment. Oct 7 05:45:16.530116119 (328h14m1.992208602s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530174690 (328h14m1.992267174s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530558932 (328h14m1.992651415s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530891485 (328h14m1.992983968s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531332224 (328h14m1.993424707s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531784496 (328h14m1.993876979s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532132368 (328h14m1.994224851s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532453750 (328h14m1.994546233s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532899228 (328h14m1.994991701s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534192571 (328h14m1.996285054s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534707100 (328h14m1.996799583s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535198395 (328h14m1.997290868s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535673079 (328h14m1.997765562s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535944537 (328h14m1.99803702s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536358194 (328h14m1.998450677s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536696960 (328h14m1.998789443s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:18.269979062 (328h14m3.732071555s) [upload=288839]: Next returned nil segment Oct 7 05:45:18.270002597 (328h14m3.73209508s) [upload=288839]: Waiting for error group managing segments... Oct 7 05:45:18.716493386 (328h14m4.178585869s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.735914286 (328h14m4.198006769s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.746153042 (328h14m4.208245535s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.764417600 (328h14m4.226510083s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.776451344 (328h14m4.238543827s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.781866084 (328h14m4.243958567s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.790674517 (328h14m4.252767s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.794717826 (328h14m4.256810319s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.823491756 (328h14m4.285584229s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:21.804432647 (328h14m7.26652513s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:27.473695541 (328h14m12.935788024s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:41.787793147 (328h14m27.249885631s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:45.508437429 (328h14m30.970529902s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508505497 (328h14m30.97059798s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508563087 (328h14m30.97065558s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508632800 (328h14m30.970725383s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508688486 (328h14m30.97078097s) [seg_pos={0 0}, upload=288839]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 05:45:45.775703629 (328h14m31.237796112s) [upload=288839]: Done uploading segments... Oct 7 05:45:45.775732604 (328h14m31.237825087s) [upload=288839]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.055246363s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.......................................................X.XX.] duration: 6.930414266s FAIL (2025-10-07 05:45:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 05:45:16.258577472 (328h14m1.720669955s) [upload=288839]: Starting upload Oct 7 05:45:16.258719863 (328h14m1.720812346s) [upload=288839]: Done starting upload Oct 7 05:45:16.259103042 (328h14m1.721195525s) [upload=288839]: Uploading segments... Oct 7 05:45:16.259160101 (328h14m1.721252584s) [upload=288839]: Got next segment. Inline: false Oct 7 05:45:16.259196861 (328h14m1.721289344s) [upload=288839]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:45:16.259235084 (328h14m1.721327567s) [upload=288839]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:45:16.530011259 (328h14m1.992103742s) [seg_pos={0 0}, upload=288839]: Begin upload segment... Oct 7 05:45:16.530086823 (328h14m1.992179306s) [seg_pos={0 0}, upload=288839]: Done begin upload segment. Oct 7 05:45:16.530116119 (328h14m1.992208602s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530174690 (328h14m1.992267174s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530558932 (328h14m1.992651415s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530891485 (328h14m1.992983968s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531332224 (328h14m1.993424707s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531784496 (328h14m1.993876979s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532132368 (328h14m1.994224851s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532453750 (328h14m1.994546233s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532899228 (328h14m1.994991701s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534192571 (328h14m1.996285054s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534707100 (328h14m1.996799583s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535198395 (328h14m1.997290868s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535673079 (328h14m1.997765562s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535944537 (328h14m1.99803702s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536358194 (328h14m1.998450677s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536696960 (328h14m1.998789443s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:18.269979062 (328h14m3.732071555s) [upload=288839]: Next returned nil segment Oct 7 05:45:18.270002597 (328h14m3.73209508s) [upload=288839]: Waiting for error group managing segments... Oct 7 05:45:18.716493386 (328h14m4.178585869s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.735914286 (328h14m4.198006769s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.746153042 (328h14m4.208245535s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.764417600 (328h14m4.226510083s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.776451344 (328h14m4.238543827s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.781866084 (328h14m4.243958567s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.790674517 (328h14m4.252767s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.794717826 (328h14m4.256810319s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.823491756 (328h14m4.285584229s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:21.804432647 (328h14m7.26652513s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:27.473695541 (328h14m12.935788024s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:41.787793147 (328h14m27.249885631s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:45.508437429 (328h14m30.970529902s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508505497 (328h14m30.97059798s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508563087 (328h14m30.97065558s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508632800 (328h14m30.970725383s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508688486 (328h14m30.97078097s) [seg_pos={0 0}, upload=288839]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 05:45:45.775703629 (328h14m31.237796112s) [upload=288839]: Done uploading segments... Oct 7 05:45:45.775732604 (328h14m31.237825087s) [upload=288839]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.930433604s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [........................................................X.XX] duration: 30.268888094s FAIL (2025-10-07 05:45:45 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 05:45:16.258577472 (328h14m1.720669955s) [upload=288839]: Starting upload Oct 7 05:45:16.258719863 (328h14m1.720812346s) [upload=288839]: Done starting upload Oct 7 05:45:16.259103042 (328h14m1.721195525s) [upload=288839]: Uploading segments... Oct 7 05:45:16.259160101 (328h14m1.721252584s) [upload=288839]: Got next segment. Inline: false Oct 7 05:45:16.259196861 (328h14m1.721289344s) [upload=288839]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:45:16.259235084 (328h14m1.721327567s) [upload=288839]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:45:16.530011259 (328h14m1.992103742s) [seg_pos={0 0}, upload=288839]: Begin upload segment... Oct 7 05:45:16.530086823 (328h14m1.992179306s) [seg_pos={0 0}, upload=288839]: Done begin upload segment. Oct 7 05:45:16.530116119 (328h14m1.992208602s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530174690 (328h14m1.992267174s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530558932 (328h14m1.992651415s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.530891485 (328h14m1.992983968s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531332224 (328h14m1.993424707s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.531784496 (328h14m1.993876979s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532132368 (328h14m1.994224851s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532453750 (328h14m1.994546233s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.532899228 (328h14m1.994991701s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534192571 (328h14m1.996285054s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.534707100 (328h14m1.996799583s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535198395 (328h14m1.997290868s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535673079 (328h14m1.997765562s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.535944537 (328h14m1.99803702s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536358194 (328h14m1.998450677s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:16.536696960 (328h14m1.998789443s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Uploading piece... Oct 7 05:45:18.269979062 (328h14m3.732071555s) [upload=288839]: Next returned nil segment Oct 7 05:45:18.270002597 (328h14m3.73209508s) [upload=288839]: Waiting for error group managing segments... Oct 7 05:45:18.716493386 (328h14m4.178585869s) [noise=true , address=45.91.116.5:4297 , piece_id=UFWWSGCNMKFLYP227JQBTPAFUWWGZHBXYG2BUTE4DZSZESTJIHCA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.735914286 (328h14m4.198006769s) [noise=true , address=77.90.31.19:4841 , piece_id=JTAZF63QUPYCASMLW6TVSQPH376URTSO2DZL2HGV7YXUZS6YBPJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.746153042 (328h14m4.208245535s) [noise=true , address=45.91.116.8:4045 , piece_id=R65HPZ7QNPYF5ROXWZS7FJPKXQJQ5BTNHWLJT6NYXEJOVBZKGXPQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.764417600 (328h14m4.226510083s) [noise=true , address=77.90.31.13:4627 , piece_id=MAM7JWL5WX6KRE2FTOKORUVSSC5WOESFB2VWPNJ5FU63BVRWJDGA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.776451344 (328h14m4.238543827s) [noise=true , address=45.91.116.11:4118 , piece_id=64A2P3IRTG6ZGAIHY3T4TVCA7KHQLFWFB7R4IRUWAIAJJXX6LCJQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.781866084 (328h14m4.243958567s) [noise=true , address=77.90.31.10:4517 , piece_id=NAU6POUA4TE7CAIUTDUA3HAF4UD7OXKCBCSPJAQNJSUSMOTR2UKQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.790674517 (328h14m4.252767s) [noise=true , address=45.91.116.4:4940 , piece_id=2XNFHUGN2PZNLZKTHXFRZVIM5RJPLRUSW75ZTXEUPUTM27RFH4YA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.794717826 (328h14m4.256810319s) [noise=true , address=45.91.116.7:4009 , piece_id=7XT5I4Z6ECW5SBANTC6IGUOJ3OT5PBXBQ7EGVQ6WIIGRXCDUE2CA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:18.823491756 (328h14m4.285584229s) [noise=true , address=77.90.31.17:4771 , piece_id=Z5FPLK6KDPIT2VXB47OW24NWHPULUVSRSRDXLWNFPCGNZZATFOFA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:21.804432647 (328h14m7.26652513s) [noise=true , address=145.79.227.2:4115 , piece_id=XPRPNT5ZRMF2FSFESM5KWVCE2BMDE2DXPMUJ4Q3XUMBHXE4SZ3GQ, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:27.473695541 (328h14m12.935788024s) [noise=true , address=145.79.227.16:4911 , piece_id=CLUVIZAOQOBPXGHMT5FODV7AAHPMFFBMPZJBSJIFZHHODQYZII7A, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:41.787793147 (328h14m27.249885631s) [noise=true , address=145.79.227.2:4114 , piece_id=H557IGOZNOARMBT4E2EF2CPRZW4IOEFPPMNP23RPXKPW4RPCXPEA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: <nil> Oct 7 05:45:45.508437429 (328h14m30.970529902s) [noise=true , address=145.79.227.15:4876 , piece_id=IYTZOAMBSQNIS6DPBGK2WFTPHH2A7KGMY6OIMOWK62HX63F3ZFXA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508505497 (328h14m30.97059798s) [noise=true , address=145.79.227.11:4732 , piece_id=32NMSXVZS6TYUN5M2MVV3FZB5TE4JPPJHBB33MCGYIYCGJ2JINIA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508563087 (328h14m30.97065558s) [noise=true , address=145.79.227.18:4005 , piece_id=VQEOCNQMJITNNLULMYCZXBVQH7ZBBSZOZW2HGULML775KULG7AVA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508632800 (328h14m30.970725383s) [noise=true , address=145.79.227.14:4837 , piece_id=DZULZKFFTUFA6T53C3KB2NBDVTO2SEDWEQ62M5DVXXSYHZGUHVLA, seg_pos={0 0}, upload=288839]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:45:45.508688486 (328h14m30.97078097s) [seg_pos={0 0}, upload=288839]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 05:45:45.775703629 (328h14m31.237796112s) [upload=288839]: Done uploading segments... Oct 7 05:45:45.775732604 (328h14m31.237825087s) [upload=288839]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.268904555s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 30.003100328s FAIL (2025-10-07 05:45:15 UTC) orbiter warning: operation: DeleteObject ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== 8h13m31.490608584s) [upload=288828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:44:46.329986961 (328h13m31.792079434s) [seg_pos={0 0}, upload=288828]: Begin upload segment... Oct 7 05:44:46.330046685 (328h13m31.792139158s) [seg_pos={0 0}, upload=288828]: Done begin upload segment. Oct 7 05:44:46.330076813 (328h13m31.792169296s) [noise=true , address=45.91.116.5:4298 , piece_id=VFN6ON6Z3CCALID52EFKZQHD6M5RKJIV2QYKR7VEEWQN4HU442AQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.333931523 (328h13m31.796023996s) [noise=true , address=77.90.31.15:4698 , piece_id=TF26GNZ5VZPVILDQL3IG6YV5O2SWBZ4ZRNYB5WTIKR3SRPJRO7KQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.335051085 (328h13m31.797143558s) [noise=true , address=145.79.227.17:4972 , piece_id=667B2LPQJHTUKDEANCFSPNJ4GBTCNV6A7N447LL3TOPGUWKHT2FA, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.336515743 (328h13m31.798608216s) [noise=true , address=145.79.227.14:4838 , piece_id=R6QCDB43VYT7QPFULJSDM3YEHVF7I5A27B7KJES7YYOTCENH7QXA, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.336655169 (328h13m31.798747652s) [noise=true , address=45.91.116.4:4938 , piece_id=IK43WVVK7PABIDUU5SJAM6M46GPS5SOHQ2DVFSIRCX2CUL7F5CGQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.337955174 (328h13m31.800047647s) [noise=true , address=145.79.227.5:4223 , piece_id=BVBUQH5SSNQLJMQEKY3L7SUS573IPGSHZOU4QTOVRMT6AYFCT5BQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.339067813 (328h13m31.801160286s) [noise=true , address=145.79.227.12:4767 , piece_id=JWIQX4WHTWCJ4GLT2OL6ZWXIQ7EL6ATY4IBVGUACZWG544COHH6Q, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.340239433 (328h13m31.802331906s) [noise=true , address=77.90.31.16:4734 , piece_id=CB6NZUOB3E2RQMTHGSOQB27P4THDHLA5YUVD5KJLZZNVCXDCVX5A, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.341381498 (328h13m31.803473991s) [noise=true , address=145.79.227.18:4005 , piece_id=I2GPTAPGJICQ4OAQERB2QBGWW2BOSBMWGMLVVGQGKMAQGEZO3CJA, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.341601107 (328h13m31.80369359s) [noise=true , address=45.91.116.13:4153 , piece_id=Z4BG3ILVXHMUDBR3WXZQDBUEJSGGRQZY2MLVI4WJ7BLSIYICNHXQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.343029346 (328h13m31.805121829s) [noise=true , address=145.79.227.16:4911 , piece_id=2F32JW3HS3QPW3INVOR2AA5PAZRDTJT77KLHPZZMUP26DSYEDRQA, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.344690489 (328h13m31.806782972s) [noise=true , address=77.90.31.13:4627 , piece_id=VCYUUPCGOZWJRMVIMBLGOXNW5AE57XIZ4RFYNO4NNVDPL4KVBVCQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.345514097 (328h13m31.80760658s) [noise=true , address=77.90.31.18:4807 , piece_id=65B3YEPSEB7WAOKB2HFQUS7N7I2FMHJDLU2WFHR7SXREZGUHJJ3A, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.347064529 (328h13m31.809157002s) [noise=true , address=45.91.116.12:4227 , piece_id=B2XA726YWIKCIF6EFDDM2PV6KIDP5JXEVG4WG5RM4DV67HM7DMVQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.348688351 (328h13m31.810780835s) [noise=true , address=145.79.227.13:4801 , piece_id=3ZXN36ZMGFNAMYBSYU7UWGCJMAMFN5MZWCAVMDYPEHBGZHPOO5HA, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:46.349742739 (328h13m31.811835222s) [noise=true , address=77.90.31.14:4661 , piece_id=F44D7PNVRQP5YVNEWUYYQRANRSWIHPUJC6XFBLZKQZPI2CO7IUWQ, seg_pos={0 0}, upload=288828]: Uploading piece... Oct 7 05:44:48.155265869 (328h13m33.617358352s) [upload=288828]: Next returned nil segment Oct 7 05:44:48.155291548 (328h13m33.617384031s) [upload=288828]: Waiting for error group managing segments... Oct 7 05:44:48.461949245 (328h13m33.924041728s) [noise=true , address=45.91.116.5:4298 , piece_id=VFN6ON6Z3CCALID52EFKZQHD6M5RKJIV2QYKR7VEEWQN4HU442AQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.473583398 (328h13m33.935675871s) [noise=true , address=45.91.116.13:4153 , piece_id=Z4BG3ILVXHMUDBR3WXZQDBUEJSGGRQZY2MLVI4WJ7BLSIYICNHXQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.499130213 (328h13m33.961222686s) [noise=true , address=77.90.31.15:4698 , piece_id=TF26GNZ5VZPVILDQL3IG6YV5O2SWBZ4ZRNYB5WTIKR3SRPJRO7KQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.502880074 (328h13m33.964972547s) [noise=true , address=77.90.31.16:4734 , piece_id=CB6NZUOB3E2RQMTHGSOQB27P4THDHLA5YUVD5KJLZZNVCXDCVX5A, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.518821718 (328h13m33.980914201s) [noise=true , address=45.91.116.4:4938 , piece_id=IK43WVVK7PABIDUU5SJAM6M46GPS5SOHQ2DVFSIRCX2CUL7F5CGQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.577335697 (328h13m34.03942817s) [noise=true , address=77.90.31.14:4661 , piece_id=F44D7PNVRQP5YVNEWUYYQRANRSWIHPUJC6XFBLZKQZPI2CO7IUWQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.593976483 (328h13m34.056068956s) [noise=true , address=77.90.31.13:4627 , piece_id=VCYUUPCGOZWJRMVIMBLGOXNW5AE57XIZ4RFYNO4NNVDPL4KVBVCQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:48.618545736 (328h13m34.080638209s) [noise=true , address=45.91.116.12:4227 , piece_id=B2XA726YWIKCIF6EFDDM2PV6KIDP5JXEVG4WG5RM4DV67HM7DMVQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:44:49.862973385 (328h13m35.325065858s) [noise=true , address=77.90.31.18:4807 , piece_id=65B3YEPSEB7WAOKB2HFQUS7N7I2FMHJDLU2WFHR7SXREZGUHJJ3A, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:45:00.690412240 (328h13m46.152504713s) [noise=true , address=145.79.227.17:4972 , piece_id=667B2LPQJHTUKDEANCFSPNJ4GBTCNV6A7N447LL3TOPGUWKHT2FA, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:45:01.857433618 (328h13m47.319526111s) [noise=true , address=145.79.227.13:4801 , piece_id=3ZXN36ZMGFNAMYBSYU7UWGCJMAMFN5MZWCAVMDYPEHBGZHPOO5HA, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:45:06.902222884 (328h13m52.364315367s) [noise=true , address=145.79.227.5:4223 , piece_id=BVBUQH5SSNQLJMQEKY3L7SUS573IPGSHZOU4QTOVRMT6AYFCT5BQ, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:45:11.215454091 (328h13m56.677546574s) [noise=true , address=145.79.227.16:4911 , piece_id=2F32JW3HS3QPW3INVOR2AA5PAZRDTJT77KLHPZZMUP26DSYEDRQA, seg_pos={0 0}, upload=288828]: Done uploading piece. err: <nil> Oct 7 05:45:11.215482195 (328h13m56.677574678s) [seg_pos={0 0}, upload=288828]: Segment reached optimal threshold of 13 pieces. Oct 7 05:45:11.215650666 (328h13m56.677743139s) [noise=true , address=145.79.227.14:4838 , piece_id=R6QCDB43VYT7QPFULJSDM3YEHVF7I5A27B7KJES7YYOTCENH7QXA, seg_pos={0 0}, upload=288828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL): protocol: write tcp 188.166.225.170:35780->145.79.227.14:4838: use of closed network connection; write tcp 188.166.225.170:35780->145.79.227.14:4838: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35780->145.79.227.14:4838: use of closed network connection Oct 7 05:45:11.215661246 (328h13m56.677753729s) [noise=true , address=145.79.227.14:4838 , piece_id=R6QCDB43VYT7QPFULJSDM3YEHVF7I5A27B7KJES7YYOTCENH7QXA, seg_pos={0 0}, upload=288828]: Long tail cancellation due to optimal threshold Oct 7 05:45:11.215793217 (328h13m56.6778857s) [noise=true , address=145.79.227.12:4767 , piece_id=JWIQX4WHTWCJ4GLT2OL6ZWXIQ7EL6ATY4IBVGUACZWG544COHH6Q, seg_pos={0 0}, upload=288828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5): protocol: write tcp 188.166.225.170:34078->145.79.227.12:4767: use of closed network connection; write tcp 188.166.225.170:34078->145.79.227.12:4767: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34078->145.79.227.12:4767: use of closed network connection Oct 7 05:45:11.215800591 (328h13m56.677893074s) [noise=true , address=145.79.227.12:4767 , piece_id=JWIQX4WHTWCJ4GLT2OL6ZWXIQ7EL6ATY4IBVGUACZWG544COHH6Q, seg_pos={0 0}, upload=288828]: Long tail cancellation due to optimal threshold Oct 7 05:45:11.215865505 (328h13m56.677957988s) [noise=true , address=145.79.227.18:4005 , piece_id=I2GPTAPGJICQ4OAQERB2QBGWW2BOSBMWGMLVVGQGKMAQGEZO3CJA, seg_pos={0 0}, upload=288828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ): protocol: write tcp 188.166.225.170:40752->145.79.227.18:4005: use of closed network connection; write tcp 188.166.225.170:40752->145.79.227.18:4005: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:40752->145.79.227.18:4005: use of closed network connection Oct 7 05:45:11.215873530 (328h13m56.677966003s) [noise=true , address=145.79.227.18:4005 , piece_id=I2GPTAPGJICQ4OAQERB2QBGWW2BOSBMWGMLVVGQGKMAQGEZO3CJA, seg_pos={0 0}, upload=288828]: Long tail cancellation due to optimal threshold Oct 7 05:45:11.215895272 (328h13m56.677987755s) [seg_pos={0 0}, upload=288828]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 05:45:11.215969413 (328h13m56.678061896s) [upload=288828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 05:45:11.215975033 (328h13m56.678067516s) [upload=288828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 05:45:11.548568176 (328h13m57.010660669s) [upload=288828]: Done uploading segments... Oct 7 05:45:11.548595147 (328h13m57.01068763s) [upload=288828]: Upload finished. err: <nil> Duration:* 30.003129344s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.421393413s FAIL (2025-10-07 05:44:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 05:43:44.130964676 (328h12m29.593057159s) [upload=288813]: Starting upload Oct 7 05:43:44.131059958 (328h12m29.593152441s) [upload=288813]: Done starting upload Oct 7 05:43:44.131267804 (328h12m29.593360287s) [upload=288813]: Uploading segments... Oct 7 05:43:44.131403021 (328h12m29.593495505s) [upload=288813]: Got next segment. Inline: false Oct 7 05:43:44.131466022 (328h12m29.593558505s) [upload=288813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:43:44.131473716 (328h12m29.593566199s) [upload=288813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:43:44.403036138 (328h12m29.865128611s) [seg_pos={0 0}, upload=288813]: Begin upload segment... Oct 7 05:43:44.403094429 (328h12m29.865186902s) [seg_pos={0 0}, upload=288813]: Done begin upload segment. Oct 7 05:43:44.403122252 (328h12m29.865214735s) [noise=true , address=145.79.227.16:4910 , piece_id=I6PLAYCNQP3NTHWNE2J3DR7ENU3ZVJTCD4DDEP4W7L2NXO6HU4UA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.403418606 (328h12m29.865511089s) [noise=true , address=145.79.227.5:4223 , piece_id=VTLRC2H3LJC5AT4W4CKOR5SSO7CHZ5D23MZJ4EFN4J7YXLZ6RCYQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.405675985 (328h12m29.867768478s) [noise=true , address=145.79.227.3:4150 , piece_id=DYVMIUP4DFHG7KMRPZMECY3U5RZJHPTDK7UCLLLHOJKWXDSI3GTA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406129588 (328h12m29.868222071s) [noise=true , address=45.91.116.13:4154 , piece_id=C33JFT2DS4DUHAC5IPGAXTEOPP5YXS2DLBC3M5OHOIGSZAIA7JJA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406404873 (328h12m29.868497366s) [noise=true , address=145.79.227.14:4840 , piece_id=6HQUITBVCNUPO6RVEPEZI74NPRBH37FEUO6F3JLWWGIB5U7354ZQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406574055 (328h12m29.868666538s) [noise=true , address=77.90.31.12:4590 , piece_id=UJICQFGQEMWFJGF7CJSPYM4NWAZG3QZLCQEJPLMNRSY2UPUJ6YVA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406925845 (328h12m29.869018318s) [noise=true , address=45.91.116.12:4226 , piece_id=JXBBL3JJDJBPAVDDSCV5VXA37RXBFGR77H3Z47DUDHM2SVVPBEHQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.407436377 (328h12m29.86952886s) [noise=true , address=77.90.31.15:4699 , piece_id=5BLJUN35HBHXEWQZJLTNORLFDMKMFRRCVY4HDQ2653REVG4ST3UQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.408021150 (328h12m29.870113633s) [noise=true , address=77.90.31.21:4879 , piece_id=IBHANU2KZ7PZ763FNJ3WSG5YB2HMXUCOZCYCD3AUOKZG4LB2B5VA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.408633126 (328h12m29.870725619s) [noise=true , address=45.91.116.8:4047 , piece_id=VE3FY3GIRD4OPT3F36HPPXYOVCNRPZ4MJ6Y4M3OQIOOHTP3I7BVQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.409258176 (328h12m29.871350659s) [noise=true , address=77.90.31.11:4553 , piece_id=WKH3LUEW7DNPYM2543GGTW6IJMIBT4NRW4E3GOYQP734D5PS5TDQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.411916227 (328h12m29.87400871s) [noise=true , address=145.79.227.15:4875 , piece_id=5AV2NXOZLYHWYBVWEQTOVBIRB3FIYYQ6FBQ4TYG37WMQ2H2B24SQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.414239982 (328h12m29.876332465s) [noise=true , address=77.90.31.14:4661 , piece_id=VC2MNI7LKNKJZUIXGGVGJ7ET3Z6GEWGBZX5SNWMQOYV55PYTVLJQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.417100479 (328h12m29.879192962s) [noise=true , address=45.91.116.6:4973 , piece_id=XM4ASJUOHHUERJM3ZER3QEYQFV7YXIEYFOZX4NN2ATJF6ZMIDG2Q, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.419649743 (328h12m29.881742226s) [noise=true , address=145.79.227.4:4186 , piece_id=QZRAXEXQFCPYAR7UBDQCI4MKS5JGQNXJFEX6SVM5IZ2BFBBUEFWQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.421966775 (328h12m29.884059258s) [noise=true , address=77.90.31.21:4878 , piece_id=RD22Q7GQFK7VPGE4HGCI2SYIOURK3ZQB7DNXC6U6HRSY2KKOUK2A, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:46.105020015 (328h12m31.567112498s) [upload=288813]: Next returned nil segment Oct 7 05:43:46.105044041 (328h12m31.567136535s) [upload=288813]: Waiting for error group managing segments... Oct 7 05:43:46.502553294 (328h12m31.964645767s) [noise=true , address=45.91.116.12:4226 , piece_id=JXBBL3JJDJBPAVDDSCV5VXA37RXBFGR77H3Z47DUDHM2SVVPBEHQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.529865379 (328h12m31.991957862s) [noise=true , address=45.91.116.13:4154 , piece_id=C33JFT2DS4DUHAC5IPGAXTEOPP5YXS2DLBC3M5OHOIGSZAIA7JJA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.538127591 (328h12m32.000220064s) [noise=true , address=77.90.31.21:4878 , piece_id=RD22Q7GQFK7VPGE4HGCI2SYIOURK3ZQB7DNXC6U6HRSY2KKOUK2A, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.550387496 (328h12m32.012479979s) [noise=true , address=45.91.116.8:4047 , piece_id=VE3FY3GIRD4OPT3F36HPPXYOVCNRPZ4MJ6Y4M3OQIOOHTP3I7BVQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.570495625 (328h12m32.032588108s) [noise=true , address=77.90.31.11:4553 , piece_id=WKH3LUEW7DNPYM2543GGTW6IJMIBT4NRW4E3GOYQP734D5PS5TDQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.593871625 (328h12m32.055964098s) [noise=true , address=45.91.116.6:4973 , piece_id=XM4ASJUOHHUERJM3ZER3QEYQFV7YXIEYFOZX4NN2ATJF6ZMIDG2Q, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.637290862 (328h12m32.099383335s) [noise=true , address=77.90.31.14:4661 , piece_id=VC2MNI7LKNKJZUIXGGVGJ7ET3Z6GEWGBZX5SNWMQOYV55PYTVLJQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.662971902 (328h12m32.125064375s) [noise=true , address=77.90.31.21:4879 , piece_id=IBHANU2KZ7PZ763FNJ3WSG5YB2HMXUCOZCYCD3AUOKZG4LB2B5VA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.668183916 (328h12m32.130276389s) [noise=true , address=77.90.31.15:4699 , piece_id=5BLJUN35HBHXEWQZJLTNORLFDMKMFRRCVY4HDQ2653REVG4ST3UQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.710971660 (328h12m32.173064153s) [noise=true , address=77.90.31.12:4590 , piece_id=UJICQFGQEMWFJGF7CJSPYM4NWAZG3QZLCQEJPLMNRSY2UPUJ6YVA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:56.175524880 (328h12m41.637617363s) [noise=true , address=145.79.227.3:4150 , piece_id=DYVMIUP4DFHG7KMRPZMECY3U5RZJHPTDK7UCLLLHOJKWXDSI3GTA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:44:08.691457926 (328h12m54.153550409s) [noise=true , address=145.79.227.16:4910 , piece_id=I6PLAYCNQP3NTHWNE2J3DR7ENU3ZVJTCD4DDEP4W7L2NXO6HU4UA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:44:13.369203185 (328h12m58.831295668s) [noise=true , address=145.79.227.4:4186 , piece_id=QZRAXEXQFCPYAR7UBDQCI4MKS5JGQNXJFEX6SVM5IZ2BFBBUEFWQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:44:13.369357519 (328h12m58.831450002s) [noise=true , address=145.79.227.14:4840 , piece_id=6HQUITBVCNUPO6RVEPEZI74NPRBH37FEUO6F3JLWWGIB5U7354ZQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52110->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52110->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52110->145.79.227.14:4840: use of closed network connection Oct 7 05:44:13.369450356 (328h12m58.831542839s) [noise=true , address=145.79.227.15:4875 , piece_id=5AV2NXOZLYHWYBVWEQTOVBIRB3FIYYQ6FBQ4TYG37WMQ2H2B24SQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:59990->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:59990->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59990->145.79.227.15:4875: use of closed network connection Oct 7 05:44:13.369528294 (328h12m58.831620777s) [noise=true , address=145.79.227.5:4223 , piece_id=VTLRC2H3LJC5AT4W4CKOR5SSO7CHZ5D23MZJ4EFN4J7YXLZ6RCYQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 188.166.225.170:36530->145.79.227.5:4223: use of closed network connection; write tcp 188.166.225.170:36530->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36530->145.79.227.5:4223: use of closed network connection Oct 7 05:44:13.369557099 (328h12m58.831649582s) [seg_pos={0 0}, upload=288813]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 05:44:13.633996337 (328h12m59.09608882s) [upload=288813]: Done uploading segments... Oct 7 05:44:13.634032857 (328h12m59.09612534s) [upload=288813]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.421415646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.267135078s FAIL (2025-10-07 05:44:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 05:43:44.130964676 (328h12m29.593057159s) [upload=288813]: Starting upload Oct 7 05:43:44.131059958 (328h12m29.593152441s) [upload=288813]: Done starting upload Oct 7 05:43:44.131267804 (328h12m29.593360287s) [upload=288813]: Uploading segments... Oct 7 05:43:44.131403021 (328h12m29.593495505s) [upload=288813]: Got next segment. Inline: false Oct 7 05:43:44.131466022 (328h12m29.593558505s) [upload=288813]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 05:43:44.131473716 (328h12m29.593566199s) [upload=288813]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 05:43:44.403036138 (328h12m29.865128611s) [seg_pos={0 0}, upload=288813]: Begin upload segment... Oct 7 05:43:44.403094429 (328h12m29.865186902s) [seg_pos={0 0}, upload=288813]: Done begin upload segment. Oct 7 05:43:44.403122252 (328h12m29.865214735s) [noise=true , address=145.79.227.16:4910 , piece_id=I6PLAYCNQP3NTHWNE2J3DR7ENU3ZVJTCD4DDEP4W7L2NXO6HU4UA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.403418606 (328h12m29.865511089s) [noise=true , address=145.79.227.5:4223 , piece_id=VTLRC2H3LJC5AT4W4CKOR5SSO7CHZ5D23MZJ4EFN4J7YXLZ6RCYQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.405675985 (328h12m29.867768478s) [noise=true , address=145.79.227.3:4150 , piece_id=DYVMIUP4DFHG7KMRPZMECY3U5RZJHPTDK7UCLLLHOJKWXDSI3GTA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406129588 (328h12m29.868222071s) [noise=true , address=45.91.116.13:4154 , piece_id=C33JFT2DS4DUHAC5IPGAXTEOPP5YXS2DLBC3M5OHOIGSZAIA7JJA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406404873 (328h12m29.868497366s) [noise=true , address=145.79.227.14:4840 , piece_id=6HQUITBVCNUPO6RVEPEZI74NPRBH37FEUO6F3JLWWGIB5U7354ZQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406574055 (328h12m29.868666538s) [noise=true , address=77.90.31.12:4590 , piece_id=UJICQFGQEMWFJGF7CJSPYM4NWAZG3QZLCQEJPLMNRSY2UPUJ6YVA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.406925845 (328h12m29.869018318s) [noise=true , address=45.91.116.12:4226 , piece_id=JXBBL3JJDJBPAVDDSCV5VXA37RXBFGR77H3Z47DUDHM2SVVPBEHQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.407436377 (328h12m29.86952886s) [noise=true , address=77.90.31.15:4699 , piece_id=5BLJUN35HBHXEWQZJLTNORLFDMKMFRRCVY4HDQ2653REVG4ST3UQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.408021150 (328h12m29.870113633s) [noise=true , address=77.90.31.21:4879 , piece_id=IBHANU2KZ7PZ763FNJ3WSG5YB2HMXUCOZCYCD3AUOKZG4LB2B5VA, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.408633126 (328h12m29.870725619s) [noise=true , address=45.91.116.8:4047 , piece_id=VE3FY3GIRD4OPT3F36HPPXYOVCNRPZ4MJ6Y4M3OQIOOHTP3I7BVQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.409258176 (328h12m29.871350659s) [noise=true , address=77.90.31.11:4553 , piece_id=WKH3LUEW7DNPYM2543GGTW6IJMIBT4NRW4E3GOYQP734D5PS5TDQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.411916227 (328h12m29.87400871s) [noise=true , address=145.79.227.15:4875 , piece_id=5AV2NXOZLYHWYBVWEQTOVBIRB3FIYYQ6FBQ4TYG37WMQ2H2B24SQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.414239982 (328h12m29.876332465s) [noise=true , address=77.90.31.14:4661 , piece_id=VC2MNI7LKNKJZUIXGGVGJ7ET3Z6GEWGBZX5SNWMQOYV55PYTVLJQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.417100479 (328h12m29.879192962s) [noise=true , address=45.91.116.6:4973 , piece_id=XM4ASJUOHHUERJM3ZER3QEYQFV7YXIEYFOZX4NN2ATJF6ZMIDG2Q, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.419649743 (328h12m29.881742226s) [noise=true , address=145.79.227.4:4186 , piece_id=QZRAXEXQFCPYAR7UBDQCI4MKS5JGQNXJFEX6SVM5IZ2BFBBUEFWQ, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:44.421966775 (328h12m29.884059258s) [noise=true , address=77.90.31.21:4878 , piece_id=RD22Q7GQFK7VPGE4HGCI2SYIOURK3ZQB7DNXC6U6HRSY2KKOUK2A, seg_pos={0 0}, upload=288813]: Uploading piece... Oct 7 05:43:46.105020015 (328h12m31.567112498s) [upload=288813]: Next returned nil segment Oct 7 05:43:46.105044041 (328h12m31.567136535s) [upload=288813]: Waiting for error group managing segments... Oct 7 05:43:46.502553294 (328h12m31.964645767s) [noise=true , address=45.91.116.12:4226 , piece_id=JXBBL3JJDJBPAVDDSCV5VXA37RXBFGR77H3Z47DUDHM2SVVPBEHQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.529865379 (328h12m31.991957862s) [noise=true , address=45.91.116.13:4154 , piece_id=C33JFT2DS4DUHAC5IPGAXTEOPP5YXS2DLBC3M5OHOIGSZAIA7JJA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.538127591 (328h12m32.000220064s) [noise=true , address=77.90.31.21:4878 , piece_id=RD22Q7GQFK7VPGE4HGCI2SYIOURK3ZQB7DNXC6U6HRSY2KKOUK2A, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.550387496 (328h12m32.012479979s) [noise=true , address=45.91.116.8:4047 , piece_id=VE3FY3GIRD4OPT3F36HPPXYOVCNRPZ4MJ6Y4M3OQIOOHTP3I7BVQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.570495625 (328h12m32.032588108s) [noise=true , address=77.90.31.11:4553 , piece_id=WKH3LUEW7DNPYM2543GGTW6IJMIBT4NRW4E3GOYQP734D5PS5TDQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.593871625 (328h12m32.055964098s) [noise=true , address=45.91.116.6:4973 , piece_id=XM4ASJUOHHUERJM3ZER3QEYQFV7YXIEYFOZX4NN2ATJF6ZMIDG2Q, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.637290862 (328h12m32.099383335s) [noise=true , address=77.90.31.14:4661 , piece_id=VC2MNI7LKNKJZUIXGGVGJ7ET3Z6GEWGBZX5SNWMQOYV55PYTVLJQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.662971902 (328h12m32.125064375s) [noise=true , address=77.90.31.21:4879 , piece_id=IBHANU2KZ7PZ763FNJ3WSG5YB2HMXUCOZCYCD3AUOKZG4LB2B5VA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.668183916 (328h12m32.130276389s) [noise=true , address=77.90.31.15:4699 , piece_id=5BLJUN35HBHXEWQZJLTNORLFDMKMFRRCVY4HDQ2653REVG4ST3UQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:46.710971660 (328h12m32.173064153s) [noise=true , address=77.90.31.12:4590 , piece_id=UJICQFGQEMWFJGF7CJSPYM4NWAZG3QZLCQEJPLMNRSY2UPUJ6YVA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:43:56.175524880 (328h12m41.637617363s) [noise=true , address=145.79.227.3:4150 , piece_id=DYVMIUP4DFHG7KMRPZMECY3U5RZJHPTDK7UCLLLHOJKWXDSI3GTA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:44:08.691457926 (328h12m54.153550409s) [noise=true , address=145.79.227.16:4910 , piece_id=I6PLAYCNQP3NTHWNE2J3DR7ENU3ZVJTCD4DDEP4W7L2NXO6HU4UA, seg_pos={0 0}, upload=288813]: Done uploading piece. err: <nil> Oct 7 05:44:13.369203185 (328h12m58.831295668s) [noise=true , address=145.79.227.4:4186 , piece_id=QZRAXEXQFCPYAR7UBDQCI4MKS5JGQNXJFEX6SVM5IZ2BFBBUEFWQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: expected piece hash; context deadline exceeded; EOF Oct 7 05:44:13.369357519 (328h12m58.831450002s) [noise=true , address=145.79.227.14:4840 , piece_id=6HQUITBVCNUPO6RVEPEZI74NPRBH37FEUO6F3JLWWGIB5U7354ZQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52110->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52110->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52110->145.79.227.14:4840: use of closed network connection Oct 7 05:44:13.369450356 (328h12m58.831542839s) [noise=true , address=145.79.227.15:4875 , piece_id=5AV2NXOZLYHWYBVWEQTOVBIRB3FIYYQ6FBQ4TYG37WMQ2H2B24SQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:1rxmCJKvyoD8UiguHRdctfRTQ1Fiu9x29M5Xbf69ihF8uBG6m2, address:145.79.227.15:4875): protocol: write tcp 188.166.225.170:59990->145.79.227.15:4875: use of closed network connection; write tcp 188.166.225.170:59990->145.79.227.15:4875: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59990->145.79.227.15:4875: use of closed network connection Oct 7 05:44:13.369528294 (328h12m58.831620777s) [noise=true , address=145.79.227.5:4223 , piece_id=VTLRC2H3LJC5AT4W4CKOR5SSO7CHZ5D23MZJ4EFN4J7YXLZ6RCYQ, seg_pos={0 0}, upload=288813]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 188.166.225.170:36530->145.79.227.5:4223: use of closed network connection; write tcp 188.166.225.170:36530->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36530->145.79.227.5:4223: use of closed network connection Oct 7 05:44:13.369557099 (328h12m58.831649582s) [seg_pos={0 0}, upload=288813]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 05:44:13.633996337 (328h12m59.09608882s) [upload=288813]: Done uploading segments... Oct 7 05:44:13.634032857 (328h12m59.09612534s) [upload=288813]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.267160255s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.095220039s FAIL (2025-10-07 03:51:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.095244355s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.970986587s FAIL (2025-10-07 03:50:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.971013839s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 6.198853974s FAIL (2025-10-07 03:50:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.198890294s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 6.173731218s FAIL (2025-10-07 03:49:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.17376397s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 5.933725415s FAIL (2025-10-07 03:48:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.933759599s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 6.199774837s FAIL (2025-10-07 03:47:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.199803942s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 6.103129801s FAIL (2025-10-07 03:47:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.10316055s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......XX....................................................] duration: 5.989361107s FAIL (2025-10-07 03:46:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.989393229s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......XX...................................................] duration: 6.020452871s FAIL (2025-10-07 03:46:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.020487146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........XX..................................................] duration: 6.099691864s FAIL (2025-10-07 03:44:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.099723545s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........XX.................................................] duration: 6.290169926s FAIL (2025-10-07 03:44:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.290206064s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........XX................................................] duration: 6.031393362s FAIL (2025-10-07 03:44:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.031427607s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........XX...............................................] duration: 6.054968401s FAIL (2025-10-07 03:43:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.054999378s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 5.890141389s FAIL (2025-10-07 03:43:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.890188468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 6.124371598s FAIL (2025-10-07 03:42:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 03:08:57.059531715 (325h37m42.521624188s) [upload=286552]: Starting upload Oct 7 03:08:57.059601327 (325h37m42.52169381s) [upload=286552]: Done starting upload Oct 7 03:08:57.059685578 (325h37m42.521778081s) [upload=286552]: Uploading segments... Oct 7 03:08:57.059776371 (325h37m42.521868844s) [upload=286552]: Got next segment. Inline: false Oct 7 03:08:57.059823220 (325h37m42.521915703s) [upload=286552]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 03:08:57.059830073 (325h37m42.521922566s) [upload=286552]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 03:08:57.397425845 (325h37m42.859518318s) [seg_pos={0 0}, upload=286552]: Begin upload segment... Oct 7 03:08:57.397495377 (325h37m42.85958786s) [seg_pos={0 0}, upload=286552]: Done begin upload segment. Oct 7 03:08:57.397516748 (325h37m42.859609221s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.397538210 (325h37m42.859630693s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398045966 (325h37m42.860138449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.398903209 (325h37m42.860995682s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399341613 (325h37m42.861434096s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399369767 (325h37m42.86146224s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399748979 (325h37m42.861841462s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.399766072 (325h37m42.861858555s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400149893 (325h37m42.862242366s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400326278 (325h37m42.862418772s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400370893 (325h37m42.862463366s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.400816472 (325h37m42.862908955s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.401729821 (325h37m42.863822314s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.403239295 (325h37m42.865331778s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404096107 (325h37m42.86618859s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:57.404801199 (325h37m42.866893692s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Uploading piece... Oct 7 03:08:59.146137129 (325h37m44.608229612s) [upload=286552]: Next returned nil segment Oct 7 03:08:59.146163560 (325h37m44.608256043s) [upload=286552]: Waiting for error group managing segments... Oct 7 03:08:59.472941344 (325h37m44.935033827s) [noise=true , address=45.91.116.14:4190 , piece_id=VHCCGOCIRKDZIQY3V65PFVLPZV6OTFDJVUZ2EV6HSFMDQ27X4JRA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.475724273 (325h37m44.937816756s) [noise=true , address=45.91.116.4:4938 , piece_id=SL6V76OEPWMQYSARCU25P5WLPY73IYXDBZVUGSMIR6OCQVYGAS4A, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.484524971 (325h37m44.946617454s) [noise=true , address=45.91.116.5:4299 , piece_id=HXER7U7HBSZH7D32SGWMC7A2YB5SUD46PRTKY2YFIB5SKEE5LSUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.489229640 (325h37m44.951322123s) [noise=true , address=45.91.116.11:4119 , piece_id=4THKRI65PNWSYIK7FHED46XADVU6NNDZT4PN7SUHAH6FMU25IWSA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.523898966 (325h37m44.985991449s) [noise=true , address=45.91.116.9:4263 , piece_id=FYRJKZSKO7K7YW7MIYLERGGGD6LVTNPMOKQBNHTZA6AYAUCWFTQA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.570530981 (325h37m45.032623464s) [noise=true , address=77.90.31.14:4661 , piece_id=ANF2SYRFIOUAAO4VPEV2LUT5LVL5L3FRF2BZNUU7DR4JRA3GO3OQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.614351321 (325h37m45.076443794s) [noise=true , address=45.91.116.6:4975 , piece_id=YB7YQSJJ6JARI3TZTY2B4DHTHYZ3ZPDNYNWM4SQZBLAJMKQ62BGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.636857456 (325h37m45.098949939s) [noise=true , address=77.90.31.16:4733 , piece_id=2JAH4P46SSYWTC6MBNZ64Y2FESOQ3XPY2PIHBTBRNLWQKGFYQKGQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.644399527 (325h37m45.106492011s) [noise=true , address=77.90.31.12:4589 , piece_id=P3HOOLP7OTFGIFMC54DGR5RGMGD72H3TS2OH52VW6XVXYSBPTBAA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.680780764 (325h37m45.142873247s) [noise=true , address=77.90.31.17:4772 , piece_id=RZQM3WORNW2VBVKSHZT2XSAAVOQC4IH7UTXWY24T2YUIGNXKIVDQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:08:59.969740736 (325h37m45.431833209s) [noise=true , address=45.91.116.7:4010 , piece_id=UDLQUBJQZC3FPWFEPHWUGFBKON52SR2TNHFKV6KQGJNILBSG5RMA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: <nil> Oct 7 03:09:26.366393109 (325h38m11.828485592s) [noise=true , address=145.79.227.4:4187 , piece_id=GULBKXEOYZGOWK4VWXADAFTVAMUS6TEAJKG2VNJUB7QVOATDKAFA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:54808->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366503389 (325h38m11.828595872s) [noise=true , address=145.79.227.13:4803 , piece_id=WF2UJ66DG3USZB3NY2JUHO64N53FGSFV32DBJKG76ZUEPVIAAFSQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:60668->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366566760 (325h38m11.828659233s) [noise=true , address=145.79.227.17:4970 , piece_id=ZK372XB5SPQFTTTZ5CYYRMTFMLVTMNHM4TOTIVVHRCVC4P472ZUQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:42880->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366658435 (325h38m11.828750908s) [noise=true , address=145.79.227.14:4838 , piece_id=ITOABMYE3UZCIRG257QSHFSIFAJPMINUOSFP4BJBFP6WBVTLJIKQ, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:53442->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366725832 (325h38m11.828818325s) [noise=true , address=145.79.227.11:4732 , piece_id=AU6HBKUEXI2LJXGMLLEVJAIE3VCN4UP7IBN5UKFLBTUE7KDJDRUA, seg_pos={0 0}, upload=286552]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:55530->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 03:09:26.366757903 (325h38m11.828850386s) [seg_pos={0 0}, upload=286552]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 03:09:26.613387934 (325h38m12.075480407s) [upload=286552]: Done uploading segments... Oct 7 03:09:26.613417010 (325h38m12.075509483s) [upload=286552]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.124396656s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)