Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.843075718s FAIL (2025-09-27 14:56:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.843106006s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 8.242320209s FAIL (2025-09-27 14:56:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.242350357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 7.3593457s FAIL (2025-09-27 14:55:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.359400676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 8.342349814s FAIL (2025-09-27 14:54:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.342380212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.751474829s FAIL (2025-09-27 14:53:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.751504607s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 7.472703719s FAIL (2025-09-27 14:52:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.47272456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.260961516s FAIL (2025-09-27 14:51:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.260994649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.934602712s FAIL (2025-09-27 14:51:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.934774429s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.74713115s FAIL (2025-09-27 14:51:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.747173761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.492647137s FAIL (2025-09-27 14:49:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.492676523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.947401725s FAIL (2025-09-27 14:49:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.947437933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.915555186s FAIL (2025-09-27 14:48:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.915590814s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.748969258s FAIL (2025-09-27 14:47:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.748996089s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 8.665275085s FAIL (2025-09-27 14:46:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.665386769s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.622392076s FAIL (2025-09-27 14:45:24 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.622425469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.441338584s FAIL (2025-09-27 14:44:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.441371146s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.607279749s FAIL (2025-09-27 14:44:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.607303083s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.869720632s FAIL (2025-09-27 14:43:30 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.869767661s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.483464801s FAIL (2025-09-27 14:42:18 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.483508165s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 11.557803837s FAIL (2025-09-27 14:41:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 11.557840086s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 9.523495381s FAIL (2025-09-27 14:40:48 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 9.523542149s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.999302158s FAIL (2025-09-27 14:40:27 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.999333026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 8.147642298s FAIL (2025-09-27 14:39:19 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.147682484s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.177367954s FAIL (2025-09-27 14:38:34 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.177399313s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 12.208423999s FAIL (2025-09-27 14:38:20 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 12.208455889s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 7.098912513s FAIL (2025-09-27 14:37:21 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.098946408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 9.218761677s FAIL (2025-09-27 14:36:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 9.218792185s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 12.776874398s FAIL (2025-09-27 14:35:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 12.776904255s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.899950492s FAIL (2025-09-27 14:34:36 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.899983777s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.895185292s FAIL (2025-09-27 14:33:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.895213996s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 10.930725552s FAIL (2025-09-27 14:32:54 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 10.930759978s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 8.186908348s FAIL (2025-09-27 14:31:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.186951049s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.962662769s FAIL (2025-09-27 14:31:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.962699651s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 7.410515747s FAIL (2025-09-27 14:30:41 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.410550493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 12.674107419s FAIL (2025-09-27 14:30:17 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 12.674143688s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 9.64784572s FAIL (2025-09-27 14:29:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 9.647878633s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.342027019s FAIL (2025-09-27 14:28:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.342061063s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.003356699s FAIL (2025-09-27 14:28:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 30.003391925s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.070517912s FAIL (2025-09-27 13:27:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.070544532s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.700908388s FAIL (2025-09-27 13:26:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.700944316s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.171799992s FAIL (2025-09-27 13:26:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.171834287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.464950693s FAIL (2025-09-27 13:25:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.464973646s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [X...X.......................................................] duration: 6.319134537s FAIL (2025-09-27 13:24:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.319175887s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.X...X......................................................] duration: 6.434154743s FAIL (2025-09-27 13:23:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.434271917s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..X...X.....................................................] duration: 6.863291376s FAIL (2025-09-27 13:22:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.863326022s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...X...X....................................................] duration: 8.490696477s FAIL (2025-09-27 13:21:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.490725183s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....X...X...................................................] duration: 6.219295276s FAIL (2025-09-27 13:21:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.219332217s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....X...X..................................................] duration: 8.390934341s FAIL (2025-09-27 13:20:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.39096533s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......X...X.................................................] duration: 7.215990091s FAIL (2025-09-27 13:20:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 7.216030919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......X...X................................................] duration: 8.830954318s FAIL (2025-09-27 13:19:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Sep 27 12:40:38.562009574 (95h9m24.024102047s) [upload=83717]: Starting upload Sep 27 12:40:38.562097762 (95h9m24.024190245s) [upload=83717]: Done starting upload Sep 27 12:40:38.562155331 (95h9m24.024247814s) [upload=83717]: Uploading segments... Sep 27 12:40:38.562413553 (95h9m24.024506036s) [upload=83717]: Got next segment. Inline: false Sep 27 12:40:38.562456485 (95h9m24.024548968s) [upload=83717]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 12:40:38.562462256 (95h9m24.024554739s) [upload=83717]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 12:40:38.809537815 (95h9m24.271630328s) [seg_pos={0 0}, upload=83717]: Begin upload segment... Sep 27 12:40:38.809611835 (95h9m24.271704308s) [seg_pos={0 0}, upload=83717]: Done begin upload segment. Sep 27 12:40:38.809640250 (95h9m24.271732723s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.809651381 (95h9m24.271743864s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810155852 (95h9m24.272248325s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.810629083 (95h9m24.272721566s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.811192586 (95h9m24.273285069s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.812766944 (95h9m24.274859427s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.813825871 (95h9m24.275918344s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.814428858 (95h9m24.276521342s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.816165556 (95h9m24.278258029s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.817752558 (95h9m24.279845041s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.820049742 (95h9m24.282142225s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.822175891 (95h9m24.284268364s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.823864976 (95h9m24.285957459s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.824678837 (95h9m24.28677132s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.825683920 (95h9m24.287776403s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:38.829240292 (95h9m24.291332776s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Uploading piece... Sep 27 12:40:40.517955198 (95h9m25.980047671s) [upload=83717]: Next returned nil segment Sep 27 12:40:40.517975026 (95h9m25.980067509s) [upload=83717]: Waiting for error group managing segments... Sep 27 12:40:40.793448910 (95h9m26.255541383s) [noise=true , address=145.79.227.3:4150 , piece_id=TYXUYRUJ5I763P6LA3ZWWJ57BAK5ERBFHGWDCCO5INCNO3TSBJLA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.809315233 (95h9m26.271407706s) [noise=true , address=145.79.227.4:4186 , piece_id=6ECNLYOYFTVMFVEOI7JPGC4N6GC73TDVMINMIBSFSX3XDJHYWJCQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.819983888 (95h9m26.282076371s) [noise=true , address=145.79.227.11:4730 , piece_id=RQ4W27NDDXYNDASF3RUFS3EHKZQGNVREPKPFCPWZO7CUXPFE32ZA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.825146579 (95h9m26.287239052s) [noise=true , address=145.79.227.19:4077 , piece_id=PZES3PIFUG5BPASSA4AB7SKQHHFFI2F2R7IUAO6IEFSOKXIYJP5Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.841531729 (95h9m26.303624212s) [noise=true , address=145.79.227.7:4293 , piece_id=JJMI533NYTNLIZ3TB5F7YTVJWSXZTVXK53ATIOP2IFGRRPWHGGNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.854031301 (95h9m26.316123774s) [noise=true , address=145.79.227.16:4909 , piece_id=L25D4RPIOKOTA5STHH7AVBV4SFNPNL4FBL4ZUR7CQJCJMJDRVS4Q, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:40.855780121 (95h9m26.317872594s) [noise=true , address=145.79.227.5:4222 , piece_id=LWZSM7MXQ5OULE6VH4E4Z7L6M4J67O3EJ4W7YMYCNPJFA7CWHQBA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.094322992 (95h9m26.556415475s) [noise=true , address=77.90.31.19:4841 , piece_id=V7QRKA6UHRPNCFWZRA2FZJLGRKJGCVFFONZZDHZQRE4H7CTZCLDA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.137933835 (95h9m26.600026318s) [noise=true , address=77.90.31.12:4590 , piece_id=7KAH42T4BT67MIRCV2G6L2CH3IHGP4NBPYD64R7VYI7552BEAAZQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:40:41.429391216 (95h9m26.891483699s) [noise=true , address=77.90.31.11:4555 , piece_id=37KARAEKEA7OCRTJJIVGZLO2KB2I4UT6FZIBW2OXRUO2OYL5JMAQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: <nil> Sep 27 12:41:07.862946034 (95h9m53.325038517s) [noise=true , address=45.91.116.10:4082 , piece_id=HTBKFN2DNSDBR66EZKQU6N3XUQNFNE5EGVVN63LGA7XVWSE5FFDQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1GwbsftFSLkxmX9qTjKAQvQ16WYTjn66q8sxCtoq7H8mvoG6Fj, address:45.91.116.10:4082): protocol: rpc: dial tcp 45.91.116.10:4082: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863078276 (95h9m53.325170759s) [noise=true , address=45.91.116.8:4048 , piece_id=TEWFOS4FVPPTHPMSCJWTQ2D5WE6TCLUXORGIHSN2HNS6U5RHODLQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB, address:45.91.116.8:4048): protocol: write tcp 188.166.225.170:55962->45.91.116.8:4048: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863119715 (95h9m53.325212208s) [noise=true , address=45.91.116.5:4298 , piece_id=XHDFZIXFNEYI35IL2Z2MQ7A537ZOZ3WH4AN7L6ULOF6VEUSDHESQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12k2VGS6TVLT1L3KSF5eUEii6VQ2jnkZaCHVrckQaCvANqFHh5C, address:45.91.116.5:4298): protocol: rpc: dial tcp 45.91.116.5:4298: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863214756 (95h9m53.325307239s) [noise=true , address=45.91.116.9:4261 , piece_id=2YAWMTOUBPONA3YYNY53LMZRPDDPDBTXSN2YCB6HDXW6A4NC6V3A, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:1eg62VsecY4ALCTk2z2TrpPFMVnfNLhNkGyUJfBfPzQrq4X8mi, address:45.91.116.9:4261): protocol: write tcp 188.166.225.170:48564->45.91.116.9:4261: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863322230 (95h9m53.325414714s) [noise=true , address=45.91.116.6:4973 , piece_id=SQOM4OVCG7OFOF3MQJNJA7JTRJW2IO6Y4HRM65P5KLXV64F2KBNQ, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:53336->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863430527 (95h9m53.32552301s) [noise=true , address=45.91.116.4:4939 , piece_id=HOVTKLVN5SQX6VFH5QZEXQZRSV6NV2D2YE5CL64HYZ6MCOB6CDHA, seg_pos={0 0}, upload=83717]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: write tcp 188.166.225.170:44714->45.91.116.4:4939: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 12:41:07.863466505 (95h9m53.325558988s) [seg_pos={0 0}, upload=83717]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Sep 27 12:41:08.104416400 (95h9m53.566508883s) [upload=83717]: Done uploading segments... Sep 27 12:41:08.104448861 (95h9m53.566541355s) [upload=83717]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 8.8309865s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)