Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.099763661s FAIL (2025-10-17 20:45:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.099809068s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.198808306s FAIL (2025-10-17 20:45:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.198840529s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 5.931377313s FAIL (2025-10-17 20:44:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.931405057s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 5.969557114s FAIL (2025-10-17 20:43:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.969594133s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 5.896731339s FAIL (2025-10-17 20:43:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.896782156s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.109038602s FAIL (2025-10-17 20:42:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.109085352s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.1472036s FAIL (2025-10-17 20:40:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.147232456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.392610926s FAIL (2025-10-17 20:40:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.392669187s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 7.905308842s FAIL (2025-10-17 20:39:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.905341935s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.074180047s FAIL (2025-10-17 20:38:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.074209613s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.279603414s FAIL (2025-10-17 20:38:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.279639724s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.496331579s FAIL (2025-10-17 20:37:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.496359773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 7.395278163s FAIL (2025-10-17 20:36:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.395327889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.358443796s FAIL (2025-10-17 20:35:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.358473383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.143570111s FAIL (2025-10-17 20:35:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.143609046s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.167979904s FAIL (2025-10-17 20:34:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.168012698s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.316491967s FAIL (2025-10-17 20:34:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.316527105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 5.955217428s FAIL (2025-10-17 20:33:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.955352154s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.195429513s FAIL (2025-10-17 20:32:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.195461404s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.134261232s FAIL (2025-10-17 20:32:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.134295759s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.142085527s FAIL (2025-10-17 20:32:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.142127548s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.132606119s FAIL (2025-10-17 20:31:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.132641799s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 7.821735679s FAIL (2025-10-17 20:30:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.821781317s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.104385949s FAIL (2025-10-17 20:29:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.104416689s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.400668125s FAIL (2025-10-17 20:28:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.400694075s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.053120151s FAIL (2025-10-17 20:27:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.053150218s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.345780709s FAIL (2025-10-17 20:27:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.345812509s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.095437024s FAIL (2025-10-17 20:25:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.095473384s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.428053908s FAIL (2025-10-17 20:25:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.4280867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.588860029s FAIL (2025-10-17 20:24:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.588897981s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 12.926223929s FAIL (2025-10-17 20:24:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 12.926277051s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.245019155s FAIL (2025-10-17 20:23:54 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== 4.456415171 (582h52m9.918507654s) [upload=512722]: Starting upload Oct 17 20:23:24.456511795 (582h52m9.918604278s) [upload=512722]: Done starting upload Oct 17 20:23:24.456649778 (582h52m9.918742271s) [upload=512722]: Uploading segments... Oct 17 20:23:24.456860238 (582h52m9.918952712s) [upload=512722]: Got next segment. Inline: false Oct 17 20:23:24.457034801 (582h52m9.919127274s) [upload=512722]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 20:23:24.457043207 (582h52m9.91913569s) [upload=512722]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 20:23:24.706826865 (582h52m10.168919358s) [seg_pos={0 0}, upload=512722]: Begin upload segment... Oct 17 20:23:24.706899794 (582h52m10.168992277s) [seg_pos={0 0}, upload=512722]: Done begin upload segment. Oct 17 20:23:24.706960229 (582h52m10.169052713s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.706975358 (582h52m10.169067841s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707567956 (582h52m10.169660439s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707747729 (582h52m10.169840212s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.707919987 (582h52m10.17001246s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708109899 (582h52m10.170202372s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708414639 (582h52m10.170507112s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708460787 (582h52m10.17055327s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708802227 (582h52m10.17089471s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.708808148 (582h52m10.170900621s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709175017 (582h52m10.17126749s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.709988686 (582h52m10.172081169s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710299318 (582h52m10.172391791s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710760326 (582h52m10.172852799s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.710993309 (582h52m10.173085792s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:24.711326224 (582h52m10.173418697s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Uploading piece... Oct 17 20:23:26.446330525 (582h52m11.908423008s) [upload=512722]: Next returned nil segment Oct 17 20:23:26.446356395 (582h52m11.908448878s) [upload=512722]: Waiting for error group managing segments... Oct 17 20:23:26.781572394 (582h52m12.243664877s) [noise=true , address=77.90.31.17:4771 , piece_id=73V6ZQLZFNR5JTME3U7L2XKZ4HV5AUDWIQEKM2DTXUUUD773T2CQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.788416577 (582h52m12.25050905s) [noise=true , address=45.91.116.6:4973 , piece_id=FBDMUV6WMV2HPEUD6KOBOXE2DLZ5MPBY4O6RDZLBPQTHNAHR42ZA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.812307771 (582h52m12.274400254s) [noise=true , address=45.91.116.9:4261 , piece_id=STSYR3NC3M764UGUDZR7Y3UFV7DZ6T7ZXZG4S5X5F4U6YX56IMXQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.823173711 (582h52m12.285266194s) [noise=true , address=77.90.31.13:4626 , piece_id=CKOIS5CFVQ2XX2NJG7MBA37BP2JQRRVOTUY73EYJVXTRIMQL56DQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.833463796 (582h52m12.295556269s) [noise=true , address=77.90.31.21:4877 , piece_id=YUGOTD4WE72F2UFYS6FX3ULWHHNGMFNDT3L3SCGUPBKABB7YFH4A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.849301915 (582h52m12.311394398s) [noise=true , address=77.90.31.12:4590 , piece_id=OHNWREW7J7EWXTIR7NFW4LAP6P6DIT7PLBKPI326YN6UJQHVNJIA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:26.869843961 (582h52m12.331936444s) [noise=true , address=77.90.31.10:4518 , piece_id=T6UONXVR5XSZNBVIERTRZ3MHZCGPZXLZ3DIFGSFBW342ZT37TZ3A, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.038985558 (582h52m24.501078031s) [noise=true , address=145.79.227.11:4730 , piece_id=KMSPAZ3H3ZKBEPW3C45PTWGRJCBK7OMN2TJVZIUQIVWH57Q7HTZQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:39.868463233 (582h52m25.330555716s) [noise=true , address=145.79.227.7:4294 , piece_id=5EDNBBJJQVTWMXZGH7DAZ7DUGLQBZYGADYGQIEZ6T7MFUE743VDA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:45.857832320 (582h52m31.319924813s) [noise=true , address=145.79.227.2:4114 , piece_id=3ZFV4FIGIBN2BFAPWSPRHR2NSYB2SYDD5JECCIHTNIBUWDRLFRTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.106183855 (582h52m38.568276328s) [noise=true , address=145.79.227.15:4875 , piece_id=Y6FZCAT2P2I4KPWLIVHBZLRPEVEJTFEUEFULDBZCCVWDDFSNPDPA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: <nil> Oct 17 20:23:53.761490272 (582h52m39.223582755s) [noise=true , address=145.79.227.5:4222 , piece_id=J7C6ZMZMYSSUVSZUZNRXE7NZ6IWCZTRSMXJO5OCN3GROJG5NAQXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:1ZLe9LXXbvrAbsYW3SKoqeihGsYhan3LgDBKjNrSspG6UnX4o1, address:145.79.227.5:4222): protocol: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45714->145.79.227.5:4222: use of closed network connection Oct 17 20:23:53.761709099 (582h52m39.223801583s) [noise=true , address=145.79.227.3:4149 , piece_id=J4UK7WJUMT3MMAHTUHR3J5PNM7SH467LGQNLQB5UWPHZ2QQPY76Q, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43910->145.79.227.3:4149: use of closed network connection Oct 17 20:23:53.761977871 (582h52m39.224070354s) [noise=true , address=145.79.227.18:4005 , piece_id=TOUMH6HNBJXUWEJDLOGDCZZ6UXIXBGCHTJLQJFLUC52MANNVDXTQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Oct 17 20:23:53.762049176 (582h52m39.224141649s) [noise=true , address=145.79.227.14:4840 , piece_id=DB33QRH4BTMZTLCKZNYF3RRRPPPTFYPT4AFTHFW6LMVVPWW4VDXA, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12crcXpNe9NgeXTsUpooLSBgrSUFbRxbSbanLa39w2Hr5mGFww, address:145.79.227.14:4840): protocol: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52782->145.79.227.14:4840: use of closed network connection Oct 17 20:23:53.762135201 (582h52m39.224227684s) [noise=true , address=145.79.227.17:4971 , piece_id=EMDMPQWD5C74POVWPO2DLDR65LZFVYUEFUFF5C5NC37XOOIMMGRQ, seg_pos={0 0}, upload=512722]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56008->145.79.227.17:4971: use of closed network connection Oct 17 20:23:53.762166841 (582h52m39.224259314s) [seg_pos={0 0}, upload=512722]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 17 20:23:54.004575256 (582h52m39.466667759s) [upload=512722]: Done uploading segments... Oct 17 20:23:54.004614520 (582h52m39.466707013s) [upload=512722]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.245049133s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.315517915s FAIL (2025-10-17 16:51:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.315547781s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.518911772s FAIL (2025-10-17 16:50:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.51895738s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.087355514s FAIL (2025-10-17 16:49:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.087389249s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.829289554s FAIL (2025-10-17 16:48:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.829329591s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.293419034s FAIL (2025-10-17 16:47:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.29344893s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.555320791s FAIL (2025-10-17 16:46:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.555407597s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.323531772s FAIL (2025-10-17 16:45:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.323564596s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.628188162s FAIL (2025-10-17 16:44:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.628220334s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.790917196s FAIL (2025-10-17 16:43:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.790949568s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 10.467102605s FAIL (2025-10-17 16:43:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, Piece ID: KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ) context deadline exceeded ======LOGS====== Oct 17 16:03:12.818514147 (578h31m58.28060662s) [upload=508912]: Starting upload Oct 17 16:03:12.818607404 (578h31m58.280699887s) [upload=508912]: Done starting upload Oct 17 16:03:12.818703758 (578h31m58.280796251s) [upload=508912]: Uploading segments... Oct 17 16:03:12.818836962 (578h31m58.280929435s) [upload=508912]: Got next segment. Inline: false Oct 17 16:03:12.818887427 (578h31m58.280979902s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 16:03:12.818894592 (578h31m58.280987075s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 16:03:13.076475108 (578h31m58.538567611s) [seg_pos={0 0}, upload=508912]: Begin upload segment... Oct 17 16:03:13.076546174 (578h31m58.538638657s) [seg_pos={0 0}, upload=508912]: Done begin upload segment. Oct 17 16:03:13.076583224 (578h31m58.538675707s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.076634282 (578h31m58.538726765s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.077612755 (578h31m58.539705238s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.078653627 (578h31m58.54074611s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079100518 (578h31m58.541193001s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079532079 (578h31m58.541624552s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079946619 (578h31m58.542039112s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081459670 (578h31m58.543552153s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081665421 (578h31m58.543757894s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083019831 (578h31m58.545112304s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083404483 (578h31m58.545496966s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.084857841 (578h31m58.546950324s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.085610945 (578h31m58.547703418s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.086781022 (578h31m58.548873505s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.089921572 (578h31m58.552014055s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.092114258 (578h31m58.554206741s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:15.679171683 (578h32m1.141264166s) [upload=508912]: Next returned nil segment Oct 17 16:03:15.679210697 (578h32m1.14130318s) [upload=508912]: Waiting for error group managing segments... Oct 17 16:03:16.096411799 (578h32m1.558504292s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.337239415 (578h32m1.799331898s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.665686374 (578h32m2.127778857s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.253070221 (578h32m2.715162704s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.592853977 (578h32m3.05494646s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.982957871 (578h32m3.445050354s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.602784996 (578h32m4.064877479s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.658692343 (578h32m4.120784856s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.938298991 (578h32m4.400391464s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.118268183 (578h32m4.580360656s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.177706804 (578h32m4.639799287s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.565276211 (578h32m5.027368695s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703915715 (578h32m5.166008198s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703955000 (578h32m5.166047473s) [seg_pos={0 0}, upload=508912]: Segment reached optimal threshold of 13 pieces. Oct 17 16:03:19.704131436 (578h32m5.166223919s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704141234 (578h32m5.166233717s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704163637 (578h32m5.16625613s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704174127 (578h32m5.1662666s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704235624 (578h32m5.166328097s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704242497 (578h32m5.16633498s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704271463 (578h32m5.166363946s) [seg_pos={0 0}, upload=508912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 17 16:03:19.704370091 (578h32m5.166462564s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:19.704376793 (578h32m5.166469266s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:20.014603007 (578h32m5.47669549s) [upload=508912]: Done uploading segments... Oct 17 16:03:20.014631049 (578h32m5.476723522s) [upload=508912]: Upload finished. err: <nil> Duration: 10.467134495s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 8.757768551s FAIL (2025-10-17 16:42:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 8.75780016s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.250294527s FAIL (2025-10-17 16:42:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.250325405s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 9.490450824s FAIL (2025-10-17 16:41:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 9.49048066s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.695531724s FAIL (2025-10-17 16:41:47 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, Piece ID: KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ) context deadline exceeded ======LOGS====== Oct 17 16:03:12.818514147 (578h31m58.28060662s) [upload=508912]: Starting upload Oct 17 16:03:12.818607404 (578h31m58.280699887s) [upload=508912]: Done starting upload Oct 17 16:03:12.818703758 (578h31m58.280796251s) [upload=508912]: Uploading segments... Oct 17 16:03:12.818836962 (578h31m58.280929435s) [upload=508912]: Got next segment. Inline: false Oct 17 16:03:12.818887427 (578h31m58.280979902s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 16:03:12.818894592 (578h31m58.280987075s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 16:03:13.076475108 (578h31m58.538567611s) [seg_pos={0 0}, upload=508912]: Begin upload segment... Oct 17 16:03:13.076546174 (578h31m58.538638657s) [seg_pos={0 0}, upload=508912]: Done begin upload segment. Oct 17 16:03:13.076583224 (578h31m58.538675707s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.076634282 (578h31m58.538726765s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.077612755 (578h31m58.539705238s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.078653627 (578h31m58.54074611s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079100518 (578h31m58.541193001s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079532079 (578h31m58.541624552s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079946619 (578h31m58.542039112s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081459670 (578h31m58.543552153s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081665421 (578h31m58.543757894s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083019831 (578h31m58.545112304s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083404483 (578h31m58.545496966s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.084857841 (578h31m58.546950324s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.085610945 (578h31m58.547703418s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.086781022 (578h31m58.548873505s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.089921572 (578h31m58.552014055s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.092114258 (578h31m58.554206741s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:15.679171683 (578h32m1.141264166s) [upload=508912]: Next returned nil segment Oct 17 16:03:15.679210697 (578h32m1.14130318s) [upload=508912]: Waiting for error group managing segments... Oct 17 16:03:16.096411799 (578h32m1.558504292s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.337239415 (578h32m1.799331898s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.665686374 (578h32m2.127778857s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.253070221 (578h32m2.715162704s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.592853977 (578h32m3.05494646s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.982957871 (578h32m3.445050354s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.602784996 (578h32m4.064877479s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.658692343 (578h32m4.120784856s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.938298991 (578h32m4.400391464s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.118268183 (578h32m4.580360656s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.177706804 (578h32m4.639799287s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.565276211 (578h32m5.027368695s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703915715 (578h32m5.166008198s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703955000 (578h32m5.166047473s) [seg_pos={0 0}, upload=508912]: Segment reached optimal threshold of 13 pieces. Oct 17 16:03:19.704131436 (578h32m5.166223919s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704141234 (578h32m5.166233717s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704163637 (578h32m5.16625613s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704174127 (578h32m5.1662666s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704235624 (578h32m5.166328097s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704242497 (578h32m5.16633498s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704271463 (578h32m5.166363946s) [seg_pos={0 0}, upload=508912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 17 16:03:19.704370091 (578h32m5.166462564s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:19.704376793 (578h32m5.166469266s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:20.014603007 (578h32m5.47669549s) [upload=508912]: Done uploading segments... Oct 17 16:03:20.014631049 (578h32m5.476723522s) [upload=508912]: Upload finished. err: <nil> Duration: 6.695560279s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.455598482s FAIL (2025-10-17 16:41:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.455736916s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.597245742s FAIL (2025-10-17 16:40:45 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, Piece ID: KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ) context deadline exceeded ======LOGS====== Oct 17 16:03:12.818514147 (578h31m58.28060662s) [upload=508912]: Starting upload Oct 17 16:03:12.818607404 (578h31m58.280699887s) [upload=508912]: Done starting upload Oct 17 16:03:12.818703758 (578h31m58.280796251s) [upload=508912]: Uploading segments... Oct 17 16:03:12.818836962 (578h31m58.280929435s) [upload=508912]: Got next segment. Inline: false Oct 17 16:03:12.818887427 (578h31m58.280979902s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 16:03:12.818894592 (578h31m58.280987075s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 16:03:13.076475108 (578h31m58.538567611s) [seg_pos={0 0}, upload=508912]: Begin upload segment... Oct 17 16:03:13.076546174 (578h31m58.538638657s) [seg_pos={0 0}, upload=508912]: Done begin upload segment. Oct 17 16:03:13.076583224 (578h31m58.538675707s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.076634282 (578h31m58.538726765s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.077612755 (578h31m58.539705238s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.078653627 (578h31m58.54074611s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079100518 (578h31m58.541193001s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079532079 (578h31m58.541624552s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079946619 (578h31m58.542039112s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081459670 (578h31m58.543552153s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081665421 (578h31m58.543757894s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083019831 (578h31m58.545112304s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083404483 (578h31m58.545496966s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.084857841 (578h31m58.546950324s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.085610945 (578h31m58.547703418s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.086781022 (578h31m58.548873505s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.089921572 (578h31m58.552014055s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.092114258 (578h31m58.554206741s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:15.679171683 (578h32m1.141264166s) [upload=508912]: Next returned nil segment Oct 17 16:03:15.679210697 (578h32m1.14130318s) [upload=508912]: Waiting for error group managing segments... Oct 17 16:03:16.096411799 (578h32m1.558504292s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.337239415 (578h32m1.799331898s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.665686374 (578h32m2.127778857s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.253070221 (578h32m2.715162704s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.592853977 (578h32m3.05494646s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.982957871 (578h32m3.445050354s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.602784996 (578h32m4.064877479s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.658692343 (578h32m4.120784856s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.938298991 (578h32m4.400391464s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.118268183 (578h32m4.580360656s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.177706804 (578h32m4.639799287s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.565276211 (578h32m5.027368695s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703915715 (578h32m5.166008198s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703955000 (578h32m5.166047473s) [seg_pos={0 0}, upload=508912]: Segment reached optimal threshold of 13 pieces. Oct 17 16:03:19.704131436 (578h32m5.166223919s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704141234 (578h32m5.166233717s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704163637 (578h32m5.16625613s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704174127 (578h32m5.1662666s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704235624 (578h32m5.166328097s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704242497 (578h32m5.16633498s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704271463 (578h32m5.166363946s) [seg_pos={0 0}, upload=508912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 17 16:03:19.704370091 (578h32m5.166462564s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:19.704376793 (578h32m5.166469266s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:20.014603007 (578h32m5.47669549s) [upload=508912]: Done uploading segments... Oct 17 16:03:20.014631049 (578h32m5.476723522s) [upload=508912]: Upload finished. err: <nil> Duration: 6.597280668s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.69296072s FAIL (2025-10-17 16:40:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== :54.423958081 (578h31m39.886050564s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424503029 (578h31m39.886595512s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.424915293 (578h31m39.887007776s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425249470 (578h31m39.887341953s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425574619 (578h31m39.887667102s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.425958911 (578h31m39.888051394s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426309318 (578h31m39.888401801s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.426649255 (578h31m39.888741738s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.428992067 (578h31m39.89108455s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.430453470 (578h31m39.892545953s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.431875066 (578h31m39.893967549s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.433277197 (578h31m39.89536968s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.434930204 (578h31m39.897022677s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.436701687 (578h31m39.89879417s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.437364319 (578h31m39.899456792s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.438985877 (578h31m39.90107836s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:54.439984658 (578h31m39.902077141s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Uploading piece... Oct 17 16:02:56.396597430 (578h31m41.858689914s) [upload=508908]: Next returned nil segment Oct 17 16:02:56.396620644 (578h31m41.858713127s) [upload=508908]: Waiting for error group managing segments... Oct 17 16:02:56.961124125 (578h31m42.423216608s) [noise=true , address=145.79.227.5:4239 , piece_id=J6E2H6LRQ45JFJCBO5K4E66XTNUGDUDPPOURWQHOIWT7MIR64RZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:57.286650792 (578h31m42.748743265s) [noise=true , address=145.79.227.26:4680 , piece_id=5CAWDM2ZNJO4OCJFB6B7S4UFEM7OHGKH5P2BA5IPWV7VHILACIWQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:58.158346779 (578h31m43.620439262s) [noise=true , address=145.79.227.35:4506 , piece_id=6XQDQHRMTV2BAO4BJXMKFSEWVK77H67KJQK6UXYPYX66GR2LGSLA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:02:59.959056615 (578h31m45.421149088s) [noise=true , address=45.91.116.5:4301 , piece_id=MGHAAQMVYWFE6VJI72SYTXNU3RDFSP3Q2S3MWNGUOAVB54N364FA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.043671056 (578h31m45.505763539s) [noise=true , address=45.91.116.10:4115 , piece_id=I5EYSDGBPZ4IBIC527P5LR2TA5YHJNBTZSPCRTSOIDINAIOYGYEA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.126873186 (578h31m45.588965669s) [noise=true , address=45.91.116.9:4280 , piece_id=WQS5V4JYQ3FGKVURNDVMPXZ6T2R4CE4Q3SWQKW2ZCBK7DLAYYUZQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.331902948 (578h31m45.793995431s) [noise=true , address=77.90.31.17:4794 , piece_id=PDPP6PA4Q7DHGZURK3YZVC7BKRBZFCWCXWO3SC6ZMPOT3GSXFIZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.439749556 (578h31m45.90184204s) [noise=true , address=77.90.31.16:4744 , piece_id=QESWRELSQSNS57LBHIB6KVS4EFMZ2YJURUXTWFJZVRKZ5W3Z2U5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.441961848 (578h31m45.904054322s) [noise=true , address=77.90.31.12:4605 , piece_id=4I2JOUUK67Y5M57OIH2WXS43POYBEQ6BX2RFJGDDW3ZZ6C2N7P4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.510011626 (578h31m45.972104109s) [noise=true , address=66.23.193.175:4554 , piece_id=HMEHVSBEXUZ4QMAAW7GLE6RFUVFU6ZCPK2ZUEZ7AVH2I6ZAZHD5Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:00.721629602 (578h31m46.183722075s) [noise=true , address=64.49.13.26:4954 , piece_id=J2QEBZKCGDBJMQN5Z2IZIC7Z4PTDQDUQ3ZOYBDOEJ6X35MPISKTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.513834375 (578h31m46.975926858s) [noise=true , address=45.91.116.8:4065 , piece_id=GEZ446UQZLV6Y6EOUXJVQQQVYJHVJNJDP6P4RB5ZCU5DGR2IKGZA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:01.799442070 (578h31m47.261534553s) [noise=true , address=64.49.13.155:4919 , piece_id=CFV5MGPS6S67MLCU5SUAM4EFLJRZEKUQHLJD7FWUVSFIOUZDWGTQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039533486 (578h31m47.501625959s) [noise=true , address=45.91.116.11:4128 , piece_id=34NL5ENKVOB4IVEY3FW5VLWLMZOMHURNPZ2JQMYO5I5NHBHRLV4Q, seg_pos={0 0}, upload=508908]: Done uploading piece. err: <nil> Oct 17 16:03:02.039580956 (578h31m47.501673439s) [seg_pos={0 0}, upload=508908]: Segment reached optimal threshold of 14 pieces. Oct 17 16:03:02.039743656 (578h31m47.501836139s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12hrM59APBvzYUrwWMvSUQVMSkzNPxaNFDWEZn2J96xYTdv4Ezm): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039759175 (578h31m47.501851658s) [noise=true , address=77.90.31.19:4871 , piece_id=W5TR5QAS3WTSOXDOIM2YWKTDSGJKLAWDFGYZEJKPEFFB2GQX6PTA, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039856300 (578h31m47.501948793s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127RqMv3h2jS5V4nVqPmi3BDwT9gdJBC93EvyA8VXf5w77yiAQP): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039870867 (578h31m47.50196336s) [noise=true , address=45.91.116.12:4230 , piece_id=2XNK3U7TQD5ILAROEFG7KWQBN4YNLO7EKQG4KKEO43EO6F4EAVKQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039876539 (578h31m47.501969022s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12faeeA3tF8s8WfxSwt75951poF8ZfeFUsEJNVXP22UeQCiBLT2): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.039887309 (578h31m47.501979792s) [noise=true , address=45.91.116.14:4207 , piece_id=JOYSYIPPHB6B4IAJUCCPSS4TJNTLS5B66TRSS3EYKMIHZ4YBMIBQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.039992550 (578h31m47.502085033s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1sZq8qBrujZ1JMFYMhfCLsjnm7V3FE2KVxPuqFeWC2VMW1yzfp): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:02.040008109 (578h31m47.502100622s) [noise=true , address=77.90.31.10:4552 , piece_id=XHKVDETTDD3JED7DDVAR4GGCQ5I356UTIUYUDRNZMIUXGRJ2VOUQ, seg_pos={0 0}, upload=508908]: Long tail cancellation due to optimal threshold Oct 17 16:03:02.040058565 (578h31m47.502151048s) [seg_pos={0 0}, upload=508908]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 17 16:03:02.040163155 (578h31m47.502255638s) [upload=508908]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:02.040172662 (578h31m47.502265145s) [upload=508908]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:02.388654651 (578h31m47.850747135s) [upload=508908]: Done uploading segments... Oct 17 16:03:02.388698234 (578h31m47.850790727s) [upload=508908]: Upload finished. err: <nil> Duration: 6.692995797s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.553841395s FAIL (2025-10-17 16:40:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 12CbfkWK4H84My3e6qhDuAkTrHWsooysz7KE4bk6VoDRDWEgb9e, Piece ID: KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ) context deadline exceeded; piecestore: (Node ID: 1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, Piece ID: VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ) context deadline exceeded ======LOGS====== Oct 17 16:03:12.818514147 (578h31m58.28060662s) [upload=508912]: Starting upload Oct 17 16:03:12.818607404 (578h31m58.280699887s) [upload=508912]: Done starting upload Oct 17 16:03:12.818703758 (578h31m58.280796251s) [upload=508912]: Uploading segments... Oct 17 16:03:12.818836962 (578h31m58.280929435s) [upload=508912]: Got next segment. Inline: false Oct 17 16:03:12.818887427 (578h31m58.280979902s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 17 16:03:12.818894592 (578h31m58.280987075s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 17 16:03:13.076475108 (578h31m58.538567611s) [seg_pos={0 0}, upload=508912]: Begin upload segment... Oct 17 16:03:13.076546174 (578h31m58.538638657s) [seg_pos={0 0}, upload=508912]: Done begin upload segment. Oct 17 16:03:13.076583224 (578h31m58.538675707s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.076634282 (578h31m58.538726765s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.077612755 (578h31m58.539705238s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.078653627 (578h31m58.54074611s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079100518 (578h31m58.541193001s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079532079 (578h31m58.541624552s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.079946619 (578h31m58.542039112s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081459670 (578h31m58.543552153s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.081665421 (578h31m58.543757894s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083019831 (578h31m58.545112304s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.083404483 (578h31m58.545496966s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.084857841 (578h31m58.546950324s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.085610945 (578h31m58.547703418s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.086781022 (578h31m58.548873505s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.089921572 (578h31m58.552014055s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:13.092114258 (578h31m58.554206741s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Uploading piece... Oct 17 16:03:15.679171683 (578h32m1.141264166s) [upload=508912]: Next returned nil segment Oct 17 16:03:15.679210697 (578h32m1.14130318s) [upload=508912]: Waiting for error group managing segments... Oct 17 16:03:16.096411799 (578h32m1.558504292s) [noise=true , address=145.79.227.13:4801 , piece_id=7XG2ESWRZVKFFKI6K5V34JHUBN4B3FIWCVOJGJMHMKIW5LNSFT6Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.337239415 (578h32m1.799331898s) [noise=true , address=145.79.227.13:4803 , piece_id=RCL4HABFHZPA2Y5SU4UPLNBHRRUJ5TS6X5VQ25K6JGFJOCWLWIKQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:16.665686374 (578h32m2.127778857s) [noise=true , address=145.79.227.7:4294 , piece_id=KDCNB6Z7KZBEM6ZJZMRSMGWHZK7S5ZSODOHDF5KMRIR4CALWPLLQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.253070221 (578h32m2.715162704s) [noise=true , address=145.79.227.5:4222 , piece_id=BWLDCTM66BZMTMGBW747OLVSNZTKU7FPMAZ5FDE77BDYLNPBR7XQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.592853977 (578h32m3.05494646s) [noise=true , address=145.79.227.2:4113 , piece_id=VRYX2VYAQ2CPA5NTZ3VPPEL4JEDMMWXOKCQPZSEOKC7EVYRY6GXQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:17.982957871 (578h32m3.445050354s) [noise=true , address=77.90.31.10:4519 , piece_id=AAVS3VSX74VLWJC5UTPD7I6B4PK2FBPHJRPNR3ODLLFYJXVLJGZA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.602784996 (578h32m4.064877479s) [noise=true , address=145.79.227.12:4767 , piece_id=MFSTDFWAMGDPRRNTPDP4LWM5MB6HQWHAZTOK7DT7TOC2JZRVZ64A, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.658692343 (578h32m4.120784856s) [noise=true , address=45.91.116.10:4083 , piece_id=IVB5SDPUHYFUNVLVUYKKSXO65ENR3CWHJ45HQTPQCE5FUCQLSCJA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:18.938298991 (578h32m4.400391464s) [noise=true , address=45.91.116.6:4974 , piece_id=I2ABZTIAGKUG5YQFVJEXQAG4BLEWI3PSIBS5VCJPZHSZ442H4TYQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.118268183 (578h32m4.580360656s) [noise=true , address=45.91.116.9:4263 , piece_id=ETBMTFT73UVYIYCJ3S4LXPJ6DQTORYOHOWMRRX6IJ5PITOAAPDSQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.177706804 (578h32m4.639799287s) [noise=true , address=45.91.116.6:4975 , piece_id=N3L2S7U7NV7DNRPEJM2ZWQLIRTJLGK52D7HN7W3YLJHTADDS6L5Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.565276211 (578h32m5.027368695s) [noise=true , address=45.91.116.11:4119 , piece_id=I7JAK4AQGEH5OZ7G7CREBLVVYWU5NFWMRF7FICQJYORQICGQBF7Q, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703915715 (578h32m5.166008198s) [noise=true , address=45.91.116.10:4082 , piece_id=CY252D7KOSDKAUKEWG6FHNYFOON6UBJ3Y24Q2DZ6BTNS23MGR7PA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: <nil> Oct 17 16:03:19.703955000 (578h32m5.166047473s) [seg_pos={0 0}, upload=508912]: Segment reached optimal threshold of 13 pieces. Oct 17 16:03:19.704131436 (578h32m5.166223919s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704141234 (578h32m5.166233717s) [noise=true , address=145.79.227.6:4259 , piece_id=GPGB6JXJCKDORACZVPMC4Y5XBTNA5K6C3OALOT4NVGZHTOXMJ5GA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704163637 (578h32m5.16625613s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dBQJ4Q1FVnvhYDRb9JrgVFPyQew4z7jfWgSsF7qxif5Mm8ai3): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704174127 (578h32m5.1662666s) [noise=true , address=77.90.31.12:4592 , piece_id=2RBR3Y72BQT26HJLXJCXWRAJX4KPYGOWCD5RREEBYDPSKBPAMIXA, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704235624 (578h32m5.166328097s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR): protocol: expected piece hash; context canceled; EOF Oct 17 16:03:19.704242497 (578h32m5.16633498s) [noise=true , address=145.79.227.16:4912 , piece_id=NKZLAAY4NQQQXYNKW677Y3LMIU5FYP7V3JVIBAR4RDK6DCDYKNJQ, seg_pos={0 0}, upload=508912]: Long tail cancellation due to optimal threshold Oct 17 16:03:19.704271463 (578h32m5.166363946s) [seg_pos={0 0}, upload=508912]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 17 16:03:19.704370091 (578h32m5.166462564s) [upload=508912]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 17 16:03:19.704376793 (578h32m5.166469266s) [upload=508912]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 17 16:03:20.014603007 (578h32m5.47669549s) [upload=508912]: Done uploading segments... Oct 17 16:03:20.014631049 (578h32m5.476723522s) [upload=508912]: Upload finished. err: <nil> Duration: 6.553878534s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)