Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.112192982s FAIL (2025-09-27 17:23:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.112220725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.084038162s FAIL (2025-09-27 17:23:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.08406856s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.45446037s FAIL (2025-09-27 17:22:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.454499193s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.340375693s FAIL (2025-09-27 17:22:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.340410939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.354062143s FAIL (2025-09-27 17:21:17 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.354094634s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.571108627s FAIL (2025-09-27 17:20:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.571148682s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.044428874s FAIL (2025-09-27 17:19:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.04446364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 5.991207086s FAIL (2025-09-27 17:19:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 5.991243265s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.317456416s FAIL (2025-09-27 17:19:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.317485522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 14.04621566s FAIL (2025-09-27 17:18:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 14.046250656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.564984015s FAIL (2025-09-27 17:17:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.565014933s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 7.074544141s FAIL (2025-09-27 17:17:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.074577775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.090616909s FAIL (2025-09-27 17:15:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.090657035s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.237451002s FAIL (2025-09-27 17:15:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.237499614s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.319198027s FAIL (2025-09-27 17:15:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.319252252s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.229628009s FAIL (2025-09-27 17:13:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 6.229662786s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.262054107s FAIL (2025-09-27 17:13:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 27 17:12:52.442224705 (99h41m37.904317189s) [upload=87713]: Starting upload Sep 27 17:12:52.442294178 (99h41m37.904386661s) [upload=87713]: Done starting upload Sep 27 17:12:52.442354312 (99h41m37.904446795s) [upload=87713]: Uploading segments... Sep 27 17:12:52.442479911 (99h41m37.904572394s) [upload=87713]: Got next segment. Inline: false Sep 27 17:12:52.442512253 (99h41m37.904604726s) [upload=87713]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 17:12:52.442518575 (99h41m37.904611058s) [upload=87713]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 17:12:52.695897035 (99h41m38.157989528s) [seg_pos={0 0}, upload=87713]: Begin upload segment... Sep 27 17:12:52.695977588 (99h41m38.158070071s) [seg_pos={0 0}, upload=87713]: Done begin upload segment. Sep 27 17:12:52.696008988 (99h41m38.158101471s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.696810224 (99h41m38.158902707s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.697576413 (99h41m38.159668896s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698025969 (99h41m38.160118452s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698347782 (99h41m38.160440255s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698673713 (99h41m38.160766196s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.698981579 (99h41m38.161074062s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699342877 (99h41m38.16143536s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.699755372 (99h41m38.161847845s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.701127334 (99h41m38.163219817s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.702625687 (99h41m38.16471817s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.703940772 (99h41m38.166033255s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.705329546 (99h41m38.167422029s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.707438873 (99h41m38.169531356s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.709442298 (99h41m38.171534781s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:52.710855087 (99h41m38.17294756s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Uploading piece... Sep 27 17:12:54.447282406 (99h41m39.909374879s) [upload=87713]: Next returned nil segment Sep 27 17:12:54.447305740 (99h41m39.909398224s) [upload=87713]: Waiting for error group managing segments... Sep 27 17:12:54.742364396 (99h41m40.204456869s) [noise=true , address=145.79.227.11:4731 , piece_id=SNZWESSPS7VLUO7KYZGQJBLGH7ZSZYBXJNXIZOCNZS2RCGMHDJBA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.757657717 (99h41m40.21975019s) [noise=true , address=145.79.227.4:4187 , piece_id=HJXJMW6EZNAUCIZESQGFX3RTCEOZFF6HXDJ44TLCTI3TLC46TPIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.758934068 (99h41m40.221026551s) [noise=true , address=145.79.227.16:4912 , piece_id=XMWBLZJP53P3MU7KKR7TVQ3ZXUW5NXAA7ZWZILL7IYM5HDO5MWCQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.762305568 (99h41m40.224398051s) [noise=true , address=145.79.227.17:4972 , piece_id=TMKGD4PKRRKYXRUWF2VOXDK4VKVXIRQY6YYG5G3ISAZQ6FEDBWIQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.767223201 (99h41m40.229315674s) [noise=true , address=145.79.227.15:4875 , piece_id=CFRXYYWIUJKIISJ5KGGOEVWN5AXZWYEIMEHFH6QKOTSTHYLLMO7A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.785463514 (99h41m40.247555997s) [noise=true , address=145.79.227.18:4005 , piece_id=XXMTE47ZUJAEYOFAA7KMGBEJLHLOXKGN7J5OVDU3NE55XPWFER5A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.848665218 (99h41m40.310757702s) [noise=true , address=145.79.227.7:4295 , piece_id=YWDMR2IBDUXJOTOMSZ3QJMIGGEH32RXMUHCTIIA4KFLJFCMSGGAQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849361765 (99h41m40.311454258s) [noise=true , address=77.90.31.18:4805 , piece_id=OG4Y65IGV7T3NWU4PEPUJBNJML2LDUMSLMKLP4WZ24LWYCFD54BA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:54.849871866 (99h41m40.31196435s) [noise=true , address=145.79.227.12:4767 , piece_id=PJ34IPNDRTXMUD3IKEMGLT7S4ZIHPS2IAYIJZRIRII5WGSK63AQA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.108840391 (99h41m40.570932864s) [noise=true , address=77.90.31.11:4553 , piece_id=FZWWTAN22AW5VCRPMPYL56QBP5I3DZCJKU2LYBJONM5GQJQDPGGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:12:55.161181898 (99h41m40.623274371s) [noise=true , address=77.90.31.13:4628 , piece_id=N7M7DAGHO76FJYS3PGXRENWN642CMFV3IIT3QINEOJ5CG6QFB6UQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: <nil> Sep 27 17:13:21.736512220 (99h42m7.198604703s) [noise=true , address=45.91.116.14:4191 , piece_id=DEBQWNBMNZD2UCPGPU6SGYSFX6K5P2OBBISDMPQZJLWYKOAVFPXQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736659290 (99h42m7.198751773s) [noise=true , address=45.91.116.5:4299 , piece_id=2CQULMDAO7HIAT5XK27T6GOKGXKNOSHJTNFJ7M2S3CWAKPXU5VIA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12SnvkmRPw7yGAfbNNx6QeZMnpo6NuiVT2PyzBAHuhgMgsxWKWX, address:45.91.116.5:4299): protocol: rpc: dial tcp 45.91.116.5:4299: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736690930 (99h42m7.198783403s) [noise=true , address=45.91.116.7:4010 , piece_id=3TZDVNFYGGWVOKHYPEHO5IVYO7GHGP62PQW3SS3VJF2J7OMUG7PQ, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:1XXhhMfCnZi1QRQeyH9L7xq346Y4bLR4m6VPAfz8fBdAtcc7w, address:45.91.116.7:4010): protocol: rpc: dial tcp 45.91.116.7:4010: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736797573 (99h42m7.198890056s) [noise=true , address=45.91.116.6:4974 , piece_id=SWFIJJ5DHICTF2SKIF2SJFQFNZV6EO2CTB3GOMFVDW4NJ5HAISGA, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12QcCaYzZBn68gjUAz268u1Hxd9apcc9F77coF8uKya1ycHdHEJ, address:45.91.116.6:4974): protocol: rpc: dial tcp 45.91.116.6:4974: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736856295 (99h42m7.198948769s) [noise=true , address=45.91.116.6:4973 , piece_id=S3QDKYOMIJCBBUOIRHSUPV775HQNYXTDUG4JLMRN4GJO52THYX3A, seg_pos={0 0}, upload=87713]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: write tcp 188.166.225.170:43610->45.91.116.6:4973: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Sep 27 17:13:21.736889869 (99h42m7.198982342s) [seg_pos={0 0}, upload=87713]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 27 17:13:21.997273891 (99h42m7.459366394s) [upload=87713]: Done uploading segments... Sep 27 17:13:21.997314718 (99h42m7.459407201s) [upload=87713]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.262080148s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 7.510766585s FAIL (2025-09-27 15:13:22 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.510799757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.752466667s FAIL (2025-09-27 15:12:43 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.752498989s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 8.316830694s FAIL (2025-09-27 15:12:26 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.316860261s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 8.078513636s FAIL (2025-09-27 15:12:08 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.078549293s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 7.114744331s FAIL (2025-09-27 15:10:52 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.114772535s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 14.973160254s FAIL (2025-09-27 15:10:23 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 14.973203958s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 7.489978898s FAIL (2025-09-27 15:09:15 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.490011119s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 14.538737979s FAIL (2025-09-27 15:08:35 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 14.538809965s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 7.545399633s FAIL (2025-09-27 15:07:28 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.545433058s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 7.949826557s FAIL (2025-09-27 15:07:05 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.949892032s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 9.424749386s FAIL (2025-09-27 15:06:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 9.424781867s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 7.767507751s FAIL (2025-09-27 15:05:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.767532808s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 7.353722879s FAIL (2025-09-27 15:04:58 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.353745532s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 10.407408579s FAIL (2025-09-27 15:04:14 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 10.40745075s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 7.427730077s FAIL (2025-09-27 15:03:09 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.427765464s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 7.638228215s FAIL (2025-09-27 15:01:55 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.638262441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.691223163s FAIL (2025-09-27 15:01:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.691252408s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 12.517445396s FAIL (2025-09-27 15:00:33 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 12.517574631s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 9.797926733s FAIL (2025-09-27 15:00:13 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 9.797953886s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 8.89054481s FAIL (2025-09-27 14:59:11 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.890575678s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 6.432413524s FAIL (2025-09-27 14:58:39 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.432448059s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 8.548826408s FAIL (2025-09-27 14:57:44 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.548862726s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 6.843075718s FAIL (2025-09-27 14:56:42 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.843106006s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 8.242320209s FAIL (2025-09-27 14:56:07 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.242350357s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 7.3593457s FAIL (2025-09-27 14:55:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.359400676s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 8.342349814s FAIL (2025-09-27 14:54:51 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 8.342380212s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.751474829s FAIL (2025-09-27 14:53:40 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.751504607s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 7.472703719s FAIL (2025-09-27 14:52:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 7.47272456s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.260961516s FAIL (2025-09-27 14:51:50 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.260994649s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.934602712s FAIL (2025-09-27 14:51:29 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.934774429s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.74713115s FAIL (2025-09-27 14:51:01 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.747173761s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.492647137s FAIL (2025-09-27 14:49:49 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.492676523s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.947401725s FAIL (2025-09-27 14:49:03 UTC) orbiter warning: operation: sync2.Copy ======ERROR====== uplink: piecestore: (Node ID: 1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, Piece ID: WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA) context deadline exceeded; piecestore: (Node ID: 1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, Piece ID: XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ) context deadline exceeded ======LOGS====== Sep 27 14:27:53.441618745 (96h56m38.903711218s) [upload=85280]: Starting upload Sep 27 14:27:53.441685943 (96h56m38.903778416s) [upload=85280]: Done starting upload Sep 27 14:27:53.441928214 (96h56m38.904020697s) [upload=85280]: Uploading segments... Sep 27 14:27:53.442016532 (96h56m38.904109015s) [upload=85280]: Got next segment. Inline: false Sep 27 14:27:53.442081045 (96h56m38.904173528s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 27 14:27:53.442086756 (96h56m38.904179239s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 27 14:27:53.747677124 (96h56m39.209769597s) [seg_pos={0 0}, upload=85280]: Begin upload segment... Sep 27 14:27:53.747730735 (96h56m39.209823208s) [seg_pos={0 0}, upload=85280]: Done begin upload segment. Sep 27 14:27:53.747754561 (96h56m39.209847034s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.747764600 (96h56m39.209857073s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748298397 (96h56m39.21039088s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.748674162 (96h56m39.210766645s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749052502 (96h56m39.211144985s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749432947 (96h56m39.21152543s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.749872324 (96h56m39.211964797s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.750296191 (96h56m39.212388664s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.751385215 (96h56m39.213477688s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.753612126 (96h56m39.215704609s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.755836862 (96h56m39.217929345s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.756623579 (96h56m39.218716063s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757224424 (96h56m39.219316907s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.757371053 (96h56m39.219463536s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768061470 (96h56m39.230153953s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:53.768947546 (96h56m39.23104004s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Uploading piece... Sep 27 14:27:55.471226618 (96h56m40.933319091s) [upload=85280]: Next returned nil segment Sep 27 14:27:55.471244733 (96h56m40.933337206s) [upload=85280]: Waiting for error group managing segments... Sep 27 14:27:55.871933798 (96h56m41.334026281s) [noise=true , address=145.79.227.4:4185 , piece_id=V4MRJIISWKATA6SMUYB6VYPSRPO4IRIR46MU7IKVIOOM5DUINZUA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.920853521 (96h56m41.382945994s) [noise=true , address=145.79.227.2:4114 , piece_id=AYLZLKWMKOU4UZCNX2UKPGNN7J4IDCCGHSU5BSSI4TUQ4MS54SWQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.923371625 (96h56m41.385464098s) [noise=true , address=77.90.31.18:4805 , piece_id=26S7TSNAVSTVTW3ID42XV25JVUM5ZQVHNBGCT6QZPH6L5EO2Z35A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:55.988913660 (96h56m41.451006133s) [noise=true , address=45.91.116.13:4154 , piece_id=Z7GYPBAYR4TNG4FWTURHVM6HZ7AUNOFH5TNIA5ULO7HMYSSMZN7A, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.034674006 (96h56m41.496766519s) [noise=true , address=45.91.116.6:4975 , piece_id=3N4OOZKLPKU5MCXODIFL67CLYUZAE7CWZN5PDX5TDHMUVZWPN2VQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.055355707 (96h56m41.51744818s) [noise=true , address=45.91.116.11:4117 , piece_id=5PEQFMMVEUA4V4IYTUKJAY3QWD6VYVEGPDH4KEDHQCHU2BBWNYLA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.070408902 (96h56m41.532501385s) [noise=true , address=77.90.31.21:4877 , piece_id=RORAAWOKD6M4ZWA2G7EVV3FRNI5OCAB2OMFKCCZMU6R7NJVQ7CJQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.098574224 (96h56m41.560666697s) [noise=true , address=45.91.116.12:4225 , piece_id=XUSCF3DVXZDLZTCX5KZPVL7PPQN4NPBD4RHAMPRU4K4NTVTOOV4Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.114161384 (96h56m41.576253867s) [noise=true , address=145.79.227.17:4970 , piece_id=XE7BASKODF7VUHK55VNYYVO2KOMI6CALCZY2GIWXRPN45MY26KQQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.213736844 (96h56m41.675829327s) [noise=true , address=45.91.116.10:4082 , piece_id=XRXMTP55UQXR6WOFPEY64QIYP32ZKC7YCPKZJBGHWXIWW7XG33ZA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.288451204 (96h56m41.750543677s) [noise=true , address=45.91.116.12:4226 , piece_id=PTU3I4Q5XNBSS3KTRAL2OOW46P7ANXTM5O3E2GTEJ5XPSPDXSHVQ, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.708832749 (96h56m42.170925222s) [noise=true , address=77.90.31.11:4554 , piece_id=3F57QM2NVD3ATD2EN7WFRSTGHCY4GRSF22RHGNE3ZCH667BWDRBA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798655717 (96h56m42.26074819s) [noise=true , address=145.79.227.7:4295 , piece_id=WXCCRTGBPJKSLY2N5X52QO2PXKSTBGHXTD6OVAZUMF2EPF4JTWEA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: <nil> Sep 27 14:27:56.798684853 (96h56m42.260777326s) [seg_pos={0 0}, upload=85280]: Segment reached optimal threshold of 13 pieces. Sep 27 14:27:56.798815943 (96h56m42.260908426s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XMLfJdbmhqnSjxRQ6DJwJowhxiqYzvi34JArdHpLzr8r2xnQ3): protocol: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60422->145.79.227.15:4873: use of closed network connection Sep 27 14:27:56.798824418 (96h56m42.260916891s) [noise=true , address=145.79.227.15:4873 , piece_id=J62C4E62YHSEOHX4JGV2D2FQSBKSZZ4RDZDCSNKLYI7AAL6CJW5Q, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798940138 (96h56m42.261032611s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798948634 (96h56m42.261041117s) [noise=true , address=145.79.227.4:4186 , piece_id=LM7YHUJ24PVKB4PSRTHTAVYJCZJ6LFBDR2TLCWP4T5CSCDAKY4EA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.798971719 (96h56m42.261064202s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK): protocol: expected piece hash; context canceled; EOF Sep 27 14:27:56.798995584 (96h56m42.261088067s) [noise=true , address=145.79.227.6:4257 , piece_id=AV5E6SXDGNL2USEDNMEUIDMJ5GHZSZYYM6HWUBIQA6OBAOCDDCWA, seg_pos={0 0}, upload=85280]: Long tail cancellation due to optimal threshold Sep 27 14:27:56.799029259 (96h56m42.261121732s) [seg_pos={0 0}, upload=85280]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 27 14:27:56.799142664 (96h56m42.261235147s) [upload=85280]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 27 14:27:56.799150429 (96h56m42.261242912s) [upload=85280]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 27 14:27:57.107487970 (96h56m42.569580453s) [upload=85280]: Done uploading segments... Sep 27 14:27:57.107551612 (96h56m42.569644095s) [upload=85280]: Upload finished. err: <nil> Duration: 6.947437933s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)