Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XXXX......] duration: 10.322076599s FAIL (2025-10-07 09:10:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 10.322107578s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XXXX.....] duration: 5.858241114s FAIL (2025-10-07 09:09:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.858274417s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [........................................X......XXXXXXXXX....] duration: 5.89104012s FAIL (2025-10-07 09:09:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 09:06:27.100979347 (331h35m12.56307183s) [upload=291794]: Starting upload Oct 7 09:06:27.101065822 (331h35m12.563158305s) [upload=291794]: Done starting upload Oct 7 09:06:27.101163388 (331h35m12.563255881s) [upload=291794]: Uploading segments... Oct 7 09:06:27.101286262 (331h35m12.563378745s) [upload=291794]: Got next segment. Inline: false Oct 7 09:06:27.101335816 (331h35m12.563428299s) [upload=291794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:06:27.101342559 (331h35m12.563435042s) [upload=291794]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:06:27.382283435 (331h35m12.844375918s) [seg_pos={0 0}, upload=291794]: Begin upload segment... Oct 7 09:06:27.382364270 (331h35m12.844456753s) [seg_pos={0 0}, upload=291794]: Done begin upload segment. Oct 7 09:06:27.382400719 (331h35m12.844493202s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382457276 (331h35m12.844549759s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382955966 (331h35m12.845048449s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383414409 (331h35m12.845506892s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383956321 (331h35m12.846048804s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384632849 (331h35m12.846725322s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384749712 (331h35m12.846842205s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385179430 (331h35m12.847271913s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385325839 (331h35m12.847418322s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385535047 (331h35m12.84762753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385907847 (331h35m12.84800033s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385989903 (331h35m12.848082386s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386246282 (331h35m12.848338765s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386588203 (331h35m12.848680686s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387052467 (331h35m12.84914494s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387813166 (331h35m12.849905639s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:29.152223721 (331h35m14.614316204s) [upload=291794]: Next returned nil segment Oct 7 09:06:29.152250412 (331h35m14.614342905s) [upload=291794]: Waiting for error group managing segments... Oct 7 09:06:29.434710130 (331h35m14.896802613s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.473092840 (331h35m14.935185313s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.504906250 (331h35m14.966998753s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.517094609 (331h35m14.979187103s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.534800766 (331h35m14.996893249s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.540557308 (331h35m15.002649792s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.561833945 (331h35m15.023926428s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.593246059 (331h35m15.055338542s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.597538133 (331h35m15.059630616s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.598174124 (331h35m15.060266607s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.606895381 (331h35m15.068987864s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.734993270 (331h35m15.197085753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:56.400160197 (331h35m41.86225267s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection Oct 7 09:06:56.400196766 (331h35m41.86228924s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection Oct 7 09:06:56.400319389 (331h35m41.862411862s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection Oct 7 09:06:56.400336182 (331h35m41.862428665s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:37394->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:56.400373333 (331h35m41.862465806s) [seg_pos={0 0}, upload=291794]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 09:06:56.640834235 (331h35m42.102926718s) [upload=291794]: Done uploading segments... Oct 7 09:06:56.640868430 (331h35m42.102960903s) [upload=291794]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 5.891075297s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XXXX....] duration: 7.116209418s FAIL (2025-10-07 09:09:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 7.116248723s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [.........................................X......XXXXXXXXX...] duration: 8.09714766s FAIL (2025-10-07 09:08:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 09:06:27.100979347 (331h35m12.56307183s) [upload=291794]: Starting upload Oct 7 09:06:27.101065822 (331h35m12.563158305s) [upload=291794]: Done starting upload Oct 7 09:06:27.101163388 (331h35m12.563255881s) [upload=291794]: Uploading segments... Oct 7 09:06:27.101286262 (331h35m12.563378745s) [upload=291794]: Got next segment. Inline: false Oct 7 09:06:27.101335816 (331h35m12.563428299s) [upload=291794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:06:27.101342559 (331h35m12.563435042s) [upload=291794]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:06:27.382283435 (331h35m12.844375918s) [seg_pos={0 0}, upload=291794]: Begin upload segment... Oct 7 09:06:27.382364270 (331h35m12.844456753s) [seg_pos={0 0}, upload=291794]: Done begin upload segment. Oct 7 09:06:27.382400719 (331h35m12.844493202s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382457276 (331h35m12.844549759s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382955966 (331h35m12.845048449s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383414409 (331h35m12.845506892s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383956321 (331h35m12.846048804s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384632849 (331h35m12.846725322s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384749712 (331h35m12.846842205s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385179430 (331h35m12.847271913s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385325839 (331h35m12.847418322s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385535047 (331h35m12.84762753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385907847 (331h35m12.84800033s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385989903 (331h35m12.848082386s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386246282 (331h35m12.848338765s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386588203 (331h35m12.848680686s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387052467 (331h35m12.84914494s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387813166 (331h35m12.849905639s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:29.152223721 (331h35m14.614316204s) [upload=291794]: Next returned nil segment Oct 7 09:06:29.152250412 (331h35m14.614342905s) [upload=291794]: Waiting for error group managing segments... Oct 7 09:06:29.434710130 (331h35m14.896802613s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.473092840 (331h35m14.935185313s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.504906250 (331h35m14.966998753s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.517094609 (331h35m14.979187103s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.534800766 (331h35m14.996893249s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.540557308 (331h35m15.002649792s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.561833945 (331h35m15.023926428s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.593246059 (331h35m15.055338542s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.597538133 (331h35m15.059630616s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.598174124 (331h35m15.060266607s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.606895381 (331h35m15.068987864s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.734993270 (331h35m15.197085753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:56.400160197 (331h35m41.86225267s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection Oct 7 09:06:56.400196766 (331h35m41.86228924s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection Oct 7 09:06:56.400319389 (331h35m41.862411862s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection Oct 7 09:06:56.400336182 (331h35m41.862428665s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:37394->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:56.400373333 (331h35m41.862465806s) [seg_pos={0 0}, upload=291794]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 09:06:56.640834235 (331h35m42.102926718s) [upload=291794]: Done uploading segments... Oct 7 09:06:56.640868430 (331h35m42.102960903s) [upload=291794]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.097177326s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [..........................................X......XXXXXXXXX..] duration: 6.1774756s FAIL (2025-10-07 09:08:16 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 09:06:27.100979347 (331h35m12.56307183s) [upload=291794]: Starting upload Oct 7 09:06:27.101065822 (331h35m12.563158305s) [upload=291794]: Done starting upload Oct 7 09:06:27.101163388 (331h35m12.563255881s) [upload=291794]: Uploading segments... Oct 7 09:06:27.101286262 (331h35m12.563378745s) [upload=291794]: Got next segment. Inline: false Oct 7 09:06:27.101335816 (331h35m12.563428299s) [upload=291794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:06:27.101342559 (331h35m12.563435042s) [upload=291794]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:06:27.382283435 (331h35m12.844375918s) [seg_pos={0 0}, upload=291794]: Begin upload segment... Oct 7 09:06:27.382364270 (331h35m12.844456753s) [seg_pos={0 0}, upload=291794]: Done begin upload segment. Oct 7 09:06:27.382400719 (331h35m12.844493202s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382457276 (331h35m12.844549759s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382955966 (331h35m12.845048449s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383414409 (331h35m12.845506892s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383956321 (331h35m12.846048804s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384632849 (331h35m12.846725322s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384749712 (331h35m12.846842205s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385179430 (331h35m12.847271913s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385325839 (331h35m12.847418322s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385535047 (331h35m12.84762753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385907847 (331h35m12.84800033s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385989903 (331h35m12.848082386s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386246282 (331h35m12.848338765s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386588203 (331h35m12.848680686s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387052467 (331h35m12.84914494s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387813166 (331h35m12.849905639s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:29.152223721 (331h35m14.614316204s) [upload=291794]: Next returned nil segment Oct 7 09:06:29.152250412 (331h35m14.614342905s) [upload=291794]: Waiting for error group managing segments... Oct 7 09:06:29.434710130 (331h35m14.896802613s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.473092840 (331h35m14.935185313s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.504906250 (331h35m14.966998753s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.517094609 (331h35m14.979187103s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.534800766 (331h35m14.996893249s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.540557308 (331h35m15.002649792s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.561833945 (331h35m15.023926428s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.593246059 (331h35m15.055338542s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.597538133 (331h35m15.059630616s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.598174124 (331h35m15.060266607s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.606895381 (331h35m15.068987864s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.734993270 (331h35m15.197085753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:56.400160197 (331h35m41.86225267s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection Oct 7 09:06:56.400196766 (331h35m41.86228924s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection Oct 7 09:06:56.400319389 (331h35m41.862411862s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection Oct 7 09:06:56.400336182 (331h35m41.862428665s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:37394->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:56.400373333 (331h35m41.862465806s) [seg_pos={0 0}, upload=291794]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 09:06:56.640834235 (331h35m42.102926718s) [upload=291794]: Done uploading segments... Oct 7 09:06:56.640868430 (331h35m42.102960903s) [upload=291794]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.177509996s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XXXX...] duration: 5.931792674s FAIL (2025-10-07 09:07:58 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.931824476s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XXXX..] duration: 5.705304297s FAIL (2025-10-07 09:07:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 5.705338202s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [...........................................X......XXXXXXXXX.] duration: 6.190605501s FAIL (2025-10-07 09:07:18 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 09:06:27.100979347 (331h35m12.56307183s) [upload=291794]: Starting upload Oct 7 09:06:27.101065822 (331h35m12.563158305s) [upload=291794]: Done starting upload Oct 7 09:06:27.101163388 (331h35m12.563255881s) [upload=291794]: Uploading segments... Oct 7 09:06:27.101286262 (331h35m12.563378745s) [upload=291794]: Got next segment. Inline: false Oct 7 09:06:27.101335816 (331h35m12.563428299s) [upload=291794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:06:27.101342559 (331h35m12.563435042s) [upload=291794]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:06:27.382283435 (331h35m12.844375918s) [seg_pos={0 0}, upload=291794]: Begin upload segment... Oct 7 09:06:27.382364270 (331h35m12.844456753s) [seg_pos={0 0}, upload=291794]: Done begin upload segment. Oct 7 09:06:27.382400719 (331h35m12.844493202s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382457276 (331h35m12.844549759s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382955966 (331h35m12.845048449s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383414409 (331h35m12.845506892s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383956321 (331h35m12.846048804s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384632849 (331h35m12.846725322s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384749712 (331h35m12.846842205s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385179430 (331h35m12.847271913s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385325839 (331h35m12.847418322s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385535047 (331h35m12.84762753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385907847 (331h35m12.84800033s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385989903 (331h35m12.848082386s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386246282 (331h35m12.848338765s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386588203 (331h35m12.848680686s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387052467 (331h35m12.84914494s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387813166 (331h35m12.849905639s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:29.152223721 (331h35m14.614316204s) [upload=291794]: Next returned nil segment Oct 7 09:06:29.152250412 (331h35m14.614342905s) [upload=291794]: Waiting for error group managing segments... Oct 7 09:06:29.434710130 (331h35m14.896802613s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.473092840 (331h35m14.935185313s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.504906250 (331h35m14.966998753s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.517094609 (331h35m14.979187103s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.534800766 (331h35m14.996893249s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.540557308 (331h35m15.002649792s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.561833945 (331h35m15.023926428s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.593246059 (331h35m15.055338542s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.597538133 (331h35m15.059630616s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.598174124 (331h35m15.060266607s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.606895381 (331h35m15.068987864s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.734993270 (331h35m15.197085753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:56.400160197 (331h35m41.86225267s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection Oct 7 09:06:56.400196766 (331h35m41.86228924s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection Oct 7 09:06:56.400319389 (331h35m41.862411862s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection Oct 7 09:06:56.400336182 (331h35m41.862428665s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:37394->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:56.400373333 (331h35m41.862465806s) [seg_pos={0 0}, upload=291794]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 09:06:56.640834235 (331h35m42.102926718s) [upload=291794]: Done uploading segments... Oct 7 09:06:56.640868430 (331h35m42.102960903s) [upload=291794]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.190641971s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 19.243893252s FAIL (2025-10-07 09:06:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 19.243910083s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 10 out of 60 is failed (threshold=3) [............................................X......XXXXXXXXX] duration: 30.24305816s FAIL (2025-10-07 09:06:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Oct 7 09:06:27.100979347 (331h35m12.56307183s) [upload=291794]: Starting upload Oct 7 09:06:27.101065822 (331h35m12.563158305s) [upload=291794]: Done starting upload Oct 7 09:06:27.101163388 (331h35m12.563255881s) [upload=291794]: Uploading segments... Oct 7 09:06:27.101286262 (331h35m12.563378745s) [upload=291794]: Got next segment. Inline: false Oct 7 09:06:27.101335816 (331h35m12.563428299s) [upload=291794]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:06:27.101342559 (331h35m12.563435042s) [upload=291794]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:06:27.382283435 (331h35m12.844375918s) [seg_pos={0 0}, upload=291794]: Begin upload segment... Oct 7 09:06:27.382364270 (331h35m12.844456753s) [seg_pos={0 0}, upload=291794]: Done begin upload segment. Oct 7 09:06:27.382400719 (331h35m12.844493202s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382457276 (331h35m12.844549759s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.382955966 (331h35m12.845048449s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383414409 (331h35m12.845506892s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.383956321 (331h35m12.846048804s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384632849 (331h35m12.846725322s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.384749712 (331h35m12.846842205s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385179430 (331h35m12.847271913s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385325839 (331h35m12.847418322s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385535047 (331h35m12.84762753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385907847 (331h35m12.84800033s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.385989903 (331h35m12.848082386s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386246282 (331h35m12.848338765s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.386588203 (331h35m12.848680686s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387052467 (331h35m12.84914494s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:27.387813166 (331h35m12.849905639s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Uploading piece... Oct 7 09:06:29.152223721 (331h35m14.614316204s) [upload=291794]: Next returned nil segment Oct 7 09:06:29.152250412 (331h35m14.614342905s) [upload=291794]: Waiting for error group managing segments... Oct 7 09:06:29.434710130 (331h35m14.896802613s) [noise=true , address=77.90.31.11:4554 , piece_id=FRA6M6UITAOJUY6HU6P7MU4KZQYCU4QUJKY5RYF265HHYE6RJWXQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.473092840 (331h35m14.935185313s) [noise=true , address=45.91.116.12:4227 , piece_id=KB5C5IR5VEUQMSODXLP6HOC2HA5GGAEQXTDKIDG52ODPBNYZF7AA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.504906250 (331h35m14.966998753s) [noise=true , address=45.91.116.13:4155 , piece_id=DBOPFGPTMYRGJVPCS3GKXOUROSXV7X3225AOOFCIXBHNDNCDR76Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.517094609 (331h35m14.979187103s) [noise=true , address=77.90.31.12:4589 , piece_id=K3JEGXYYL6QRCMWUGIQ5SG5YYBRI5AKXHBXRRQQH4L3VBDD3TNBA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.534800766 (331h35m14.996893249s) [noise=true , address=45.91.116.11:4118 , piece_id=G423VMWPKBWAYRW6R2S7NXNTEOY2752UNR6L362X2TS2ZJCWR3OA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.540557308 (331h35m15.002649792s) [noise=true , address=77.90.31.14:4663 , piece_id=BMMEMJM7WHLX6WSFJ7IZFBFD4NCY6FEHNGDMUUOS3YVJHBJWP4KQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.561833945 (331h35m15.023926428s) [noise=true , address=45.91.116.8:4045 , piece_id=YRACXLU2U6MV3G3PLPXEMMR2HJWJ4MXSLZUJRMQEDQFK3OULAZWA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.593246059 (331h35m15.055338542s) [noise=true , address=45.91.116.4:4940 , piece_id=VO4OVM5RWOD5MNDYJBR6VZNDRRI4WXZKJTEMBNZTP76OYTKHD7LQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.597538133 (331h35m15.059630616s) [noise=true , address=45.91.116.11:4119 , piece_id=4UOVS5SCSAI6DRHIPO5FY5OMRMEBDMOCJYO7CG455Z3JS4FBVUIA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.598174124 (331h35m15.060266607s) [noise=true , address=45.91.116.7:4011 , piece_id=JGAECHNLHHJDPXZGTBYE6OBGVFF6IZCTKXFTNAYMG2HR5MSVAIEA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.606895381 (331h35m15.068987864s) [noise=true , address=45.91.116.14:4189 , piece_id=FRIRTK5EEBIUBD2HWFTYMADEJFMJWRLRRNGCTPOWAM7AG2QK5N7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:29.734993270 (331h35m15.197085753s) [noise=true , address=77.90.31.10:4520 , piece_id=7EKSWYA4TQWHWWU2IF6IQUWCBOGK3NHUZEZEVCTVACJI3HWETO7Q, seg_pos={0 0}, upload=291794]: Done uploading piece. err: <nil> Oct 7 09:06:56.400160197 (331h35m41.86225267s) [noise=true , address=145.79.227.17:4969 , piece_id=XNBHWB73DSW4GWGIBLBN5C2SFBYHOFE67RZQS43W62HH3XC5GDGA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59490->145.79.227.17:4969: use of closed network connection Oct 7 09:06:56.400196766 (331h35m41.86228924s) [noise=true , address=145.79.227.6:4258 , piece_id=UBIWA4EDYE3QUOSD27AIGFQZI6C6ZKR4U4FIIGTSLVZYUKT4AMMQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52930->145.79.227.6:4258: use of closed network connection Oct 7 09:06:56.400319389 (331h35m41.862411862s) [noise=true , address=145.79.227.15:4874 , piece_id=PH7JKO5WVKMKO4FZU3DFGZOMRDRGKWUHUB3ZPOI4Q7TCYBXBFMXA, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46722->145.79.227.15:4874: use of closed network connection Oct 7 09:06:56.400336182 (331h35m41.862428665s) [noise=true , address=145.79.227.7:4295 , piece_id=36QJ3O2MYTTQO6QX4MYK5OTJNRMA6TJXTEHICU6L3QYPLU26TTRQ, seg_pos={0 0}, upload=291794]: Done uploading piece. err: ecclient: upload failed (node:1kjzBTGEGEs1qKgbkvvC9bBfPrUa6r6yToz1qhS1pjVqEagauT, address:145.79.227.7:4295): protocol: read tcp 188.166.225.170:37394->145.79.227.7:4295: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:56.400373333 (331h35m41.862465806s) [seg_pos={0 0}, upload=291794]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Oct 7 09:06:56.640834235 (331h35m42.102926718s) [upload=291794]: Done uploading segments... Oct 7 09:06:56.640868430 (331h35m42.102960903s) [upload=291794]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.243086495s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 1m0.247086791s FAIL (2025-10-07 09:06:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 13) ======LOGS====== 22s) [seg_pos={0 0}, upload=291781]: Done begin upload segment. Oct 7 09:05:38.674342299 (331h34m24.136434782s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677465517 (331h34m24.13955799s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.677936033 (331h34m24.140028616s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678118630 (331h34m24.140211114s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678568848 (331h34m24.140661331s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.678944493 (331h34m24.141036966s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679001381 (331h34m24.141093864s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679458753 (331h34m24.141551236s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.679888300 (331h34m24.141980783s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680050439 (331h34m24.142142913s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.680343277 (331h34m24.14243575s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.681078477 (331h34m24.14317098s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.684727766 (331h34m24.146820249s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.685431045 (331h34m24.147523528s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.689654407 (331h34m24.15174689s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.690866004 (331h34m24.152958488s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.691563192 (331h34m24.153655675s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:38.693214215 (331h34m24.155306709s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Uploading piece... Oct 7 09:05:40.011854369 (331h34m25.473946852s) [upload=291781]: Next returned nil segment Oct 7 09:05:40.011874477 (331h34m25.47396695s) [upload=291781]: Waiting for error group managing segments... Oct 7 09:05:40.359699787 (331h34m25.82179226s) [noise=true , address=64.49.13.26:4950 , piece_id=LNINIXGZWF3OUCERM74PMNTIBZI4VZFYXHD5OTB6PZDE7FBVZ5RA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.408932707 (331h34m25.87102519s) [noise=true , address=64.49.13.137:4501 , piece_id=CVMZCT5UW6HRRFAWMSU6L2TWQYIKKFHZISLEZTEG3PAX5GGYRZSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.629949103 (331h34m26.092041586s) [noise=true , address=45.91.116.5:4331 , piece_id=K2J43VMYVPMPEHQ7CTGIRLNIJFKJMBF77LYYZNM6XYY7MDEE7X7A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.636435575 (331h34m26.098528058s) [noise=true , address=77.90.31.21:4906 , piece_id=3JTE2UXYXJ55XTE7KP4UVJFGDKWM66AZQ5X53XNSOPWBLDLYHMSA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.645030461 (331h34m26.107122944s) [noise=true , address=77.90.31.16:4765 , piece_id=4XJWWBPTTOPGYZKVYZFDTJRGJKTYWFUEDRL4DGZSSYI2J4FVZKBA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.686868922 (331h34m26.148961405s) [noise=true , address=45.91.116.12:4235 , piece_id=LRUGENH7JPZQT27HJWM5A7WVKNPV43NALSHNN67XAZ4WCIB2UVOQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.696859636 (331h34m26.158952109s) [noise=true , address=45.91.116.4:4958 , piece_id=TZGNMVMVMYD54CIHYM3FM5YNFOB2NPBOBDMA3N3RDDGMCLOZHRFQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.731516541 (331h34m26.193609024s) [noise=true , address=77.90.31.12:4594 , piece_id=GQEU7LCAFJLUOSDAFTZHHI3BSNB3L72HI57YSSMDZERLI2UXXETQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.761145401 (331h34m26.223237884s) [noise=true , address=45.91.116.13:4178 , piece_id=IGXJUAKE5NAWOK5ERWUP7LPPME2P44LVXWGLGZCTBBYK35FN3Q3A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.777550841 (331h34m26.239643324s) [noise=true , address=77.90.31.11:4582 , piece_id=DQCXYW75WZLFOTQJN76BSLKAD3L3DQ4QDGVJP5AHS3JYWLHXM66A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.782368435 (331h34m26.244460918s) [noise=true , address=77.90.31.13:4625 , piece_id=6IW7GNEDNLUDFEVUCQEU2MJP2M52Y22ZLTCD2QDUYDECE65Q5KUA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.801133097 (331h34m26.26322558s) [noise=true , address=45.91.116.10:4086 , piece_id=56XS7LZ5FMYBWSFBX7XWGQZDVJHE6LX2VO4D2ILWPM7AJESGVE3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:05:40.866507564 (331h34m26.328600047s) [noise=true , address=77.90.31.10:4528 , piece_id=ICC3CERVBMM4YNATUXG3IGOUXEUFKUWINS533LVPS6QFFULN3FOA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: <nil> Oct 7 09:06:37.718010366 (331h35m23.180102849s) [noise=true , address=145.79.227.27:4715 , piece_id=Y77777EHRMOMJ4MYC2UC4Y6GDBJLWMAWCMO67QGKVAJ2LCBT5IWA, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dtsPeGJbDSsaR1rPCFE42aCvH8o6AsnRMSK8Dp2RmcpFxo3CR, address:145.79.227.27:4715): protocol: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60246->145.79.227.27:4715: use of closed network connection Oct 7 09:06:37.718129113 (331h35m23.180221596s) [noise=true , address=145.79.227.33:4881 , piece_id=VAXI4I4MYXVNONIMZYX7STOJ46NTZE5AV2WJTE2USNE7FDNK5M3Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1mMBjg1b4MqTZPE5NpNNMYDzmsHqWYotE5jBNmZLyA5FyYLZRp, address:145.79.227.33:4881): protocol: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60486->145.79.227.33:4881: use of closed network connection Oct 7 09:06:37.718269180 (331h35m23.180361673s) [noise=true , address=145.79.227.49:4338 , piece_id=4EQ2XHI5EQHNZUAQZ7XBSQG2V57YV6CPECNO4OG2DRC3ZUPNUN2Q, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1cTadUV1Zu2Z8MoobXvyZ965fv6jPFhYGsy5qgRZHgWeSwyitN, address:145.79.227.49:4338): protocol: read tcp 188.166.225.170:56276->145.79.227.49:4338: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:37.718320016 (331h35m23.180412489s) [noise=true , address=145.79.227.42:4136 , piece_id=EQAOXE4DXMPLCFCRFX2YXNDCC55WL2WANW2GA67IASXJG4WFUBHQ, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:12rkSxE1mCbyoUyh3pPwz6xdw8rXqpeyFrhciFoB8bgs4TkKmyT, address:145.79.227.42:4136): protocol: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52874->145.79.227.42:4136: use of closed network connection Oct 7 09:06:37.718488457 (331h35m23.18058094s) [noise=true , address=145.79.227.31:4808 , piece_id=CSC6KZCVVBZWUPNDLZHXV27GRK47IHUHV2Q4LGKOSX442MNH6O5A, seg_pos={0 0}, upload=291781]: Done uploading piece. err: ecclient: upload failed (node:1dBGnojgbYNwFfATQTahFBnkvZCGyZLiwRBa3AYv4iVY4U3yE2, address:145.79.227.31:4808): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:06:37.718535526 (331h35m23.180628009s) [seg_pos={0 0}, upload=291781]: Done waiting for segment. successful: 13 optimal: 14 errs: <nil> Oct 7 09:06:37.963123606 (331h35m23.425216089s) [upload=291781]: Done uploading segments... Oct 7 09:06:37.963154384 (331h35m23.425246858s) [upload=291781]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 13) Duration: 1m0.247101618s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 9 out of 60 is failed (threshold=3) [.............................................X......XXXXXXXX] duration: 30.255973403s FAIL (2025-10-07 09:06:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== ments... Oct 7 09:05:42.351535729 (331h34m27.813628202s) [upload=291782]: Got next segment. Inline: false Oct 7 09:05:42.351599340 (331h34m27.813691823s) [upload=291782]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:05:42.351608227 (331h34m27.8137007s) [upload=291782]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:05:42.597595663 (331h34m28.059688156s) [seg_pos={0 0}, upload=291782]: Begin upload segment... Oct 7 09:05:42.597670245 (331h34m28.059762728s) [seg_pos={0 0}, upload=291782]: Done begin upload segment. Oct 7 09:05:42.597699621 (331h34m28.059792094s) [noise=true , address=77.90.31.18:4805 , piece_id=7BWLMNOAQXZQXCDZT2XQSFNM57FNUIHYYP3WMH4FQELAFNVVKSTA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.597766258 (331h34m28.059858731s) [noise=true , address=45.91.116.11:4117 , piece_id=UJ4OWZF5JEH3VUQLCM2E7LVMJQTTYWRDGL5XG3AZOHJOOKZOK23A, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.598409293 (331h34m28.060501776s) [noise=true , address=145.79.227.16:4910 , piece_id=PM6UP4EFLLAGBF4YCYZIRBAOK52RRJ7STH5M2Z2KLSPQ3N4UPTRA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.598851204 (331h34m28.060943677s) [noise=true , address=77.90.31.21:4879 , piece_id=IES5XT64IBDW5Q65V7JJTBAEVL7EGBYT7Z2Q3ULOFUPYZ4TLHWUA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.600051119 (331h34m28.062143602s) [noise=true , address=45.91.116.7:4010 , piece_id=UB5N3TUVUYOEOM4ZKTDSGPQ3ER4C6ICXWHKPUQBBDOLHEUXHJLIQ, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.600498070 (331h34m28.062590553s) [noise=true , address=45.91.116.4:4939 , piece_id=KBOZM4ZLF6QGIKN4PEXTJCPGGQJUVYBAHEQLPFVXQMAIO6I254EA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.601078145 (331h34m28.063170628s) [noise=true , address=145.79.227.14:4837 , piece_id=4ZLHDGUPDIQVGXIJRCO2MZQQQLVV4K56IDBUVJLB34AE3WSTDE7Q, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.601511500 (331h34m28.063603973s) [noise=true , address=77.90.31.12:4589 , piece_id=WWIHOPFQ23G4PCSR3DEK3UR5ZKFGYT4J4RD45JMVIRVKTGKNN4UQ, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.603064277 (331h34m28.06515676s) [noise=true , address=145.79.227.4:4187 , piece_id=L7LTUTSHJZLKRBYSACOZ7KDSPTXDZE4TWBL2MAV24ERPO5V6X7VA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.603263356 (331h34m28.065355839s) [noise=true , address=77.90.31.15:4698 , piece_id=64HAOVFXHJWMRWMNCB3KVDMDIPMW5MJRWFNBNFBIVERPTG7ONYLQ, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.605108249 (331h34m28.067200722s) [noise=true , address=77.90.31.19:4843 , piece_id=TSNKANBNOWZPV3BBKV4T4YFKVA72MSGDNASCUUJNZXJIRMUAJDTQ, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.605189845 (331h34m28.067282328s) [noise=true , address=145.79.227.17:4971 , piece_id=KTBALMDALI3XQFGWIEEXFB7Y3QRC6O767IBOXWZ2MRFUZCBJ3PAQ, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.607150858 (331h34m28.069243351s) [noise=true , address=145.79.227.13:4801 , piece_id=TEP3JUGKQTUFGO2VYJUQY7F47GXIDH27KTJKNT522TVNF6XEMH2Q, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.608848641 (331h34m28.070941114s) [noise=true , address=45.91.116.10:4083 , piece_id=ZCMOKFTO35BJE3YTSJPMQTTC6BJJAAGSNTQYHOB5YLWE7YAW6MWA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.610582042 (331h34m28.072674525s) [noise=true , address=77.90.31.11:4553 , piece_id=GGEFFBKIBAVTCYIY367G6VSHYV4SI55LUQS5U4JQFESQW4BQCIUA, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:42.612348336 (331h34m28.074440819s) [noise=true , address=145.79.227.4:4186 , piece_id=VLGMSD52KIG3MPCS4K5EG4QS25GQXOH47P5GCDLXVXE7WNQ73HBQ, seg_pos={0 0}, upload=291782]: Uploading piece... Oct 7 09:05:44.386870451 (331h34m29.848962934s) [upload=291782]: Next returned nil segment Oct 7 09:05:44.386894998 (331h34m29.848987471s) [upload=291782]: Waiting for error group managing segments... Oct 7 09:05:44.667080175 (331h34m30.129172658s) [noise=true , address=45.91.116.7:4010 , piece_id=UB5N3TUVUYOEOM4ZKTDSGPQ3ER4C6ICXWHKPUQBBDOLHEUXHJLIQ, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.683210360 (331h34m30.145302843s) [noise=true , address=45.91.116.11:4117 , piece_id=UJ4OWZF5JEH3VUQLCM2E7LVMJQTTYWRDGL5XG3AZOHJOOKZOK23A, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.766846923 (331h34m30.228939406s) [noise=true , address=45.91.116.4:4939 , piece_id=KBOZM4ZLF6QGIKN4PEXTJCPGGQJUVYBAHEQLPFVXQMAIO6I254EA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.766986680 (331h34m30.229079153s) [noise=true , address=77.90.31.19:4843 , piece_id=TSNKANBNOWZPV3BBKV4T4YFKVA72MSGDNASCUUJNZXJIRMUAJDTQ, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.817233171 (331h34m30.279325654s) [noise=true , address=77.90.31.11:4553 , piece_id=GGEFFBKIBAVTCYIY367G6VSHYV4SI55LUQS5U4JQFESQW4BQCIUA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.824260432 (331h34m30.286352905s) [noise=true , address=77.90.31.18:4805 , piece_id=7BWLMNOAQXZQXCDZT2XQSFNM57FNUIHYYP3WMH4FQELAFNVVKSTA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.824719175 (331h34m30.286811658s) [noise=true , address=77.90.31.15:4698 , piece_id=64HAOVFXHJWMRWMNCB3KVDMDIPMW5MJRWFNBNFBIVERPTG7ONYLQ, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.878381221 (331h34m30.340473704s) [noise=true , address=77.90.31.21:4879 , piece_id=IES5XT64IBDW5Q65V7JJTBAEVL7EGBYT7Z2Q3ULOFUPYZ4TLHWUA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.885179526 (331h34m30.347272009s) [noise=true , address=45.91.116.10:4083 , piece_id=ZCMOKFTO35BJE3YTSJPMQTTC6BJJAAGSNTQYHOB5YLWE7YAW6MWA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:05:44.910484843 (331h34m30.372577326s) [noise=true , address=77.90.31.12:4589 , piece_id=WWIHOPFQ23G4PCSR3DEK3UR5ZKFGYT4J4RD45JMVIRVKTGKNN4UQ, seg_pos={0 0}, upload=291782]: Done uploading piece. err: <nil> Oct 7 09:06:11.655664649 (331h34m57.117757132s) [noise=true , address=145.79.227.13:4801 , piece_id=TEP3JUGKQTUFGO2VYJUQY7F47GXIDH27KTJKNT522TVNF6XEMH2Q, seg_pos={0 0}, upload=291782]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:41322->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:41322->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:41322->145.79.227.13:4801: use of closed network connection Oct 7 09:06:11.655886092 (331h34m57.117978575s) [noise=true , address=145.79.227.16:4910 , piece_id=PM6UP4EFLLAGBF4YCYZIRBAOK52RRJ7STH5M2Z2KLSPQ3N4UPTRA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: write tcp 188.166.225.170:57754->145.79.227.16:4910: use of closed network connection; write tcp 188.166.225.170:57754->145.79.227.16:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57754->145.79.227.16:4910: use of closed network connection Oct 7 09:06:11.655976513 (331h34m57.118068996s) [noise=true , address=145.79.227.17:4971 , piece_id=KTBALMDALI3XQFGWIEEXFB7Y3QRC6O767IBOXWZ2MRFUZCBJ3PAQ, seg_pos={0 0}, upload=291782]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: write tcp 188.166.225.170:49430->145.79.227.17:4971: use of closed network connection; write tcp 188.166.225.170:49430->145.79.227.17:4971: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:49430->145.79.227.17:4971: use of closed network connection Oct 7 09:06:11.656216290 (331h34m57.118308773s) [noise=true , address=145.79.227.14:4837 , piece_id=4ZLHDGUPDIQVGXIJRCO2MZQQQLVV4K56IDBUVJLB34AE3WSTDE7Q, seg_pos={0 0}, upload=291782]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 188.166.225.170:56052->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:11.656349885 (331h34m57.118442368s) [noise=true , address=145.79.227.4:4186 , piece_id=VLGMSD52KIG3MPCS4K5EG4QS25GQXOH47P5GCDLXVXE7WNQ73HBQ, seg_pos={0 0}, upload=291782]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: read tcp 188.166.225.170:41562->145.79.227.4:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:11.656541169 (331h34m57.118633652s) [noise=true , address=145.79.227.4:4187 , piece_id=L7LTUTSHJZLKRBYSACOZ7KDSPTXDZE4TWBL2MAV24ERPO5V6X7VA, seg_pos={0 0}, upload=291782]: Done uploading piece. err: ecclient: upload failed (node:12cQxBB2L2CyymYx3xWp1ALj6p8QWr4A34sqvtG5zyeT7sVWv7x, address:145.79.227.4:4187): protocol: read tcp 188.166.225.170:46066->145.79.227.4:4187: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:06:11.656572409 (331h34m57.118664892s) [seg_pos={0 0}, upload=291782]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:06:11.910175588 (331h34m57.372268071s) [upload=291782]: Done uploading segments... Oct 7 09:06:11.910199112 (331h34m57.372291595s) [upload=291782]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.255986668s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 8 out of 60 is failed (threshold=3) [..............................................X......XXXXXXX] duration: 30.242109189s FAIL (2025-10-07 09:05:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== 6051 (331h33m57.327148544s) [upload=291776]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:05:11.865063506 (331h33m57.327155989s) [upload=291776]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:05:12.113679757 (331h33m57.57577224s) [seg_pos={0 0}, upload=291776]: Begin upload segment... Oct 7 09:05:12.113758618 (331h33m57.575851101s) [seg_pos={0 0}, upload=291776]: Done begin upload segment. Oct 7 09:05:12.113793965 (331h33m57.575886448s) [noise=true , address=45.91.116.4:4937 , piece_id=3BJPK6FDOOI6LQY74VUAMEVMAKOIW54AOU4NKEOHVIFGPL7EYQTA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.113870521 (331h33m57.575963004s) [noise=true , address=77.90.31.18:4805 , piece_id=5KEGP3LXC4A5AAURLIJMYMYCFJJWO6GAOE2WTVNZON2OM2UZ6LJQ, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.114505009 (331h33m57.576597482s) [noise=true , address=145.79.227.5:4223 , piece_id=EE2ZAE3SACKJJ5WE57UH7RNA523K5QEK2MRWZHNZYT55MRQKAYKQ, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.114924077 (331h33m57.57701656s) [noise=true , address=77.90.31.10:4520 , piece_id=YAACF2DTFXVBLDMHSS454HVFAD3VVEIAK7TGNJLQMHUQW45EYRZQ, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.115238145 (331h33m57.577330628s) [noise=true , address=145.79.227.2:4113 , piece_id=BZ3I46A7KFWISO46H35EQNZKJ5HY5X5EYRMET5JCOTL3XMBMP2HA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.115652915 (331h33m57.577745398s) [noise=true , address=145.79.227.14:4837 , piece_id=JB44BFHXW3NAFQINECROVBDKFVPFMJQSQVLXXPSOFYHUYICZLK4A, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.116836228 (331h33m57.578928721s) [noise=true , address=145.79.227.16:4909 , piece_id=7Q57NFL5ZDV3KCMIHSDYQDQZGOVS4Z4L7IEPER65KSIMLTYEBDGA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.117387929 (331h33m57.579480412s) [noise=true , address=45.91.116.8:4047 , piece_id=OI5S3ZYYYLN36AIQ7IN46ZBA2KH7PP2NWSPOSPCUHQN7YRYMZDUA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.117993432 (331h33m57.580085915s) [noise=true , address=145.79.227.4:4186 , piece_id=RPBSGFMB6P4N2432ORQJM5672K5TLNFBOZBEHYQXAFREESFPRZOA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.118536586 (331h33m57.580629069s) [noise=true , address=77.90.31.19:4843 , piece_id=HJT4O4OGG5S75CCMJIX56UH6QI4IUJET3NMBOGC4QT3ENTNPV6IA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.119023283 (331h33m57.581115756s) [noise=true , address=45.91.116.7:4011 , piece_id=YZXXS2G3PD4H63BVHB7MRJUIMPTWTSFNI5V7CHVTDORR6UQNDNFQ, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.120023668 (331h33m57.582116141s) [noise=true , address=145.79.227.15:4876 , piece_id=RSPRAX7XD2FLBH7HP3ZJOG2M2FCZZVOLACGQN3U3C53DO2Z3CVHQ, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.121274570 (331h33m57.583367053s) [noise=true , address=77.90.31.17:4772 , piece_id=T6YL5EHEIQTYIK33NW6VL6HRORZCM2FBLKKSH7B2N7SQ6GGRZURA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.123569971 (331h33m57.585662454s) [noise=true , address=45.91.116.9:4262 , piece_id=C3Y5OT3NSNYE5MDU3D2I34FKJRCDZ7DWNUIS45GM2T5FH4NRWVDA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.124322073 (331h33m57.586414546s) [noise=true , address=145.79.227.17:4972 , piece_id=QABXZ4ENOQ4LQUQ26YIDC2ABR3SHAFDBP2MHLJS5PQHBE57AUZXQ, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:12.126516021 (331h33m57.588608504s) [noise=true , address=77.90.31.12:4589 , piece_id=NUFX7M5IZL3L3Y3CXRJROP5WSDQI3NCD6K4RCKOVZW6RYJXJPEUA, seg_pos={0 0}, upload=291776]: Uploading piece... Oct 7 09:05:13.825775331 (331h33m59.287867814s) [upload=291776]: Next returned nil segment Oct 7 09:05:13.825848600 (331h33m59.287941073s) [upload=291776]: Waiting for error group managing segments... Oct 7 09:05:14.228928027 (331h33m59.69102051s) [noise=true , address=77.90.31.17:4772 , piece_id=T6YL5EHEIQTYIK33NW6VL6HRORZCM2FBLKKSH7B2N7SQ6GGRZURA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.234290017 (331h33m59.69638249s) [noise=true , address=77.90.31.18:4805 , piece_id=5KEGP3LXC4A5AAURLIJMYMYCFJJWO6GAOE2WTVNZON2OM2UZ6LJQ, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.250147402 (331h33m59.712239885s) [noise=true , address=45.91.116.9:4262 , piece_id=C3Y5OT3NSNYE5MDU3D2I34FKJRCDZ7DWNUIS45GM2T5FH4NRWVDA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.269450039 (331h33m59.731542512s) [noise=true , address=77.90.31.12:4589 , piece_id=NUFX7M5IZL3L3Y3CXRJROP5WSDQI3NCD6K4RCKOVZW6RYJXJPEUA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.283847216 (331h33m59.745939699s) [noise=true , address=45.91.116.4:4937 , piece_id=3BJPK6FDOOI6LQY74VUAMEVMAKOIW54AOU4NKEOHVIFGPL7EYQTA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.297795155 (331h33m59.759887628s) [noise=true , address=77.90.31.19:4843 , piece_id=HJT4O4OGG5S75CCMJIX56UH6QI4IUJET3NMBOGC4QT3ENTNPV6IA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.313026719 (331h33m59.775119192s) [noise=true , address=45.91.116.7:4011 , piece_id=YZXXS2G3PD4H63BVHB7MRJUIMPTWTSFNI5V7CHVTDORR6UQNDNFQ, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.328526595 (331h33m59.790619078s) [noise=true , address=77.90.31.10:4520 , piece_id=YAACF2DTFXVBLDMHSS454HVFAD3VVEIAK7TGNJLQMHUQW45EYRZQ, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:14.339525098 (331h33m59.801617581s) [noise=true , address=45.91.116.8:4047 , piece_id=OI5S3ZYYYLN36AIQ7IN46ZBA2KH7PP2NWSPOSPCUHQN7YRYMZDUA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: <nil> Oct 7 09:05:41.167585005 (331h34m26.629677488s) [noise=true , address=145.79.227.14:4837 , piece_id=JB44BFHXW3NAFQINECROVBDKFVPFMJQSQVLXXPSOFYHUYICZLK4A, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:37790->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:37790->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37790->145.79.227.14:4837: use of closed network connection Oct 7 09:05:41.167704332 (331h34m26.629796816s) [noise=true , address=145.79.227.2:4113 , piece_id=BZ3I46A7KFWISO46H35EQNZKJ5HY5X5EYRMET5JCOTL3XMBMP2HA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: read tcp 188.166.225.170:53600->145.79.227.2:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:41.167713009 (331h34m26.629805492s) [noise=true , address=145.79.227.16:4909 , piece_id=7Q57NFL5ZDV3KCMIHSDYQDQZGOVS4Z4L7IEPER65KSIMLTYEBDGA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: write tcp 188.166.225.170:43110->145.79.227.16:4909: use of closed network connection; write tcp 188.166.225.170:43110->145.79.227.16:4909: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:43110->145.79.227.16:4909: use of closed network connection Oct 7 09:05:41.167826975 (331h34m26.629919448s) [noise=true , address=145.79.227.17:4972 , piece_id=QABXZ4ENOQ4LQUQ26YIDC2ABR3SHAFDBP2MHLJS5PQHBE57AUZXQ, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:12natVZCpE83KmTNWBmNV6FGRy62n2HtcodmHwyDwpoDaP8qVBX, address:145.79.227.17:4972): protocol: read tcp 188.166.225.170:49032->145.79.227.17:4972: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:41.167913050 (331h34m26.630005523s) [noise=true , address=145.79.227.5:4223 , piece_id=EE2ZAE3SACKJJ5WE57UH7RNA523K5QEK2MRWZHNZYT55MRQKAYKQ, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: read tcp 188.166.225.170:55732->145.79.227.5:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:41.167984706 (331h34m26.630077189s) [noise=true , address=145.79.227.15:4876 , piece_id=RSPRAX7XD2FLBH7HP3ZJOG2M2FCZZVOLACGQN3U3C53DO2Z3CVHQ, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:47482->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:41.168059889 (331h34m26.630152362s) [noise=true , address=145.79.227.4:4186 , piece_id=RPBSGFMB6P4N2432ORQJM5672K5TLNFBOZBEHYQXAFREESFPRZOA, seg_pos={0 0}, upload=291776]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: read tcp 188.166.225.170:49722->145.79.227.4:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: context deadline exceeded Oct 7 09:05:41.168087272 (331h34m26.630179745s) [seg_pos={0 0}, upload=291776]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 09:05:41.408029166 (331h34m26.870121649s) [upload=291776]: Done uploading segments... Oct 7 09:05:41.408059283 (331h34m26.870151756s) [upload=291776]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.242124449s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 1m0.249592728s FAIL (2025-10-07 09:05:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 11) ======LOGS====== noise=true , address=145.79.227.25:4646 , piece_id=CNB4NVHI657PMBJ5ZZGOXRAV2F6E26C5RHNC33FQLLBNEC74S65Q, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.166917658 (331h33m23.629010141s) [noise=true , address=145.79.227.26:4679 , piece_id=225VDGLHHSPTDXYW2AZES72KQ2PXBI5WWRHL4PY4KEVKAPYCL4EA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.167361914 (331h33m23.629454387s) [noise=true , address=45.91.116.13:4159 , piece_id=6APN763S6KZMJETD2PJHQNLMDR4YI5QW4J2IHOTXDPQ2S4QCZSCQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.167705068 (331h33m23.629797541s) [noise=true , address=45.91.116.11:4139 , piece_id=CV4KAL5S4DWLPT4QLWHLBPWJMNBIGS2N5ZCC2VEDOZPZGXWERYBQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.168025518 (331h33m23.630118001s) [noise=true , address=145.79.227.30:4786 , piece_id=7SWSXZX4US5UCDSEFVW3FYSAJ2VMX3U6NRAPF2P36F74DDGXDVJQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.168359214 (331h33m23.630451687s) [noise=true , address=145.79.227.36:4552 , piece_id=DYBRNQP2EXHG6OPOSHUBWDZECJQEEABNITRASLEXH6KYG4QN5OGQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.168793251 (331h33m23.630885724s) [noise=true , address=77.90.31.14:4664 , piece_id=D22OTVY3QAV2XTNC4OB7MVOSQL727PIQIZR6HVEI23WH2TQZGFJQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.169109553 (331h33m23.631202026s) [noise=true , address=45.91.116.12:4253 , piece_id=TYLHWIPBCDWA622RXVSHQB6K3JBIVEOLTEUPGUS72B72RRNLQQKA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.169505577 (331h33m23.63159805s) [noise=true , address=45.91.116.14:4215 , piece_id=HHL3KRA2II25X4ARM3LKIL55MYAOAL2WM3PQFQFBM5L6DDHZR5MA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.169967306 (331h33m23.632059779s) [noise=true , address=77.90.31.19:4868 , piece_id=Z3HFMNKGIZP5KZT3NGXFT4FD7S5VRSFH4AGYCYT4W4L35Y3VT2BQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.171677592 (331h33m23.633770065s) [noise=true , address=77.90.31.12:4593 , piece_id=B2RQLRXNX2YSUQJQCCRVMH2QD7S67X3TU4GJ3OCFSAAE7BN6FXFQ, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.172099536 (331h33m23.63419201s) [noise=true , address=145.79.227.34:4931 , piece_id=YSG743WSKWN4CQPZINNXCLI2C7YKZ3KOETMPY2GK4CBAN54XXGPA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.172504507 (331h33m23.63459698s) [noise=true , address=77.90.31.16:4761 , piece_id=TPU45OMGFTCFESRHCSYQ7ZPQO6HMHPLDOF3TIV77HV5TZK73633A, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.172722423 (331h33m23.634814936s) [noise=true , address=77.90.31.21:4891 , piece_id=XT3EE7XQBVWDVBJRMLZAJLZ7TJF2YBG3OTPL4CB434YOTS5MA6XA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.173233405 (331h33m23.635325878s) [noise=true , address=145.79.227.41:4094 , piece_id=UPQTDEMR2V34ZCREBNBFCZSGPU7PEXIASX6VYPX5KBSRJKFP4HBA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:38.173586557 (331h33m23.63567903s) [noise=true , address=145.79.227.33:4900 , piece_id=P7YO24SJ5XVS7HOZFBRPGQ4KKNIT5OKIJVGCD3E3L5HPSFH7MBCA, seg_pos={0 0}, upload=291765]: Uploading piece... Oct 7 09:04:39.740836942 (331h33m25.202929435s) [upload=291765]: Next returned nil segment Oct 7 09:04:39.740857270 (331h33m25.202949754s) [upload=291765]: Waiting for error group managing segments... Oct 7 09:04:40.146825949 (331h33m25.608918422s) [noise=true , address=77.90.31.19:4868 , piece_id=Z3HFMNKGIZP5KZT3NGXFT4FD7S5VRSFH4AGYCYT4W4L35Y3VT2BQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.149721202 (331h33m25.611813675s) [noise=true , address=45.91.116.12:4253 , piece_id=TYLHWIPBCDWA622RXVSHQB6K3JBIVEOLTEUPGUS72B72RRNLQQKA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.168412454 (331h33m25.630504937s) [noise=true , address=77.90.31.12:4593 , piece_id=B2RQLRXNX2YSUQJQCCRVMH2QD7S67X3TU4GJ3OCFSAAE7BN6FXFQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.176833519 (331h33m25.638926002s) [noise=true , address=45.91.116.7:4042 , piece_id=A64P25GRPHSARP45D2HQTXIXUEJWVDOVM3HI47GMHCUWYLZ7YIZQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.202448517 (331h33m25.664541s) [noise=true , address=77.90.31.21:4891 , piece_id=XT3EE7XQBVWDVBJRMLZAJLZ7TJF2YBG3OTPL4CB434YOTS5MA6XA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.219658138 (331h33m25.681750621s) [noise=true , address=77.90.31.14:4664 , piece_id=D22OTVY3QAV2XTNC4OB7MVOSQL727PIQIZR6HVEI23WH2TQZGFJQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.228980619 (331h33m25.691073102s) [noise=true , address=45.91.116.11:4139 , piece_id=CV4KAL5S4DWLPT4QLWHLBPWJMNBIGS2N5ZCC2VEDOZPZGXWERYBQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.229756436 (331h33m25.69184896s) [noise=true , address=45.91.116.14:4215 , piece_id=HHL3KRA2II25X4ARM3LKIL55MYAOAL2WM3PQFQFBM5L6DDHZR5MA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.243300157 (331h33m25.70539263s) [noise=true , address=45.91.116.13:4159 , piece_id=6APN763S6KZMJETD2PJHQNLMDR4YI5QW4J2IHOTXDPQ2S4QCZSCQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.245274577 (331h33m25.70736705s) [noise=true , address=77.90.31.16:4761 , piece_id=TPU45OMGFTCFESRHCSYQ7ZPQO6HMHPLDOF3TIV77HV5TZK73633A, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:04:40.266632708 (331h33m25.728725181s) [noise=true , address=77.90.31.17:4792 , piece_id=OWZYMYATH2X362MSQHZAWUIZ3DMS6GDFUNUVMLB5N5U3GNXXSX2A, seg_pos={0 0}, upload=291765]: Done uploading piece. err: <nil> Oct 7 09:05:37.205108321 (331h34m22.667200794s) [noise=true , address=145.79.227.41:4094 , piece_id=UPQTDEMR2V34ZCREBNBFCZSGPU7PEXIASX6VYPX5KBSRJKFP4HBA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:1QYpeobqZ5QtLcuNBh9fHhiKxnbEtpiAaagvgWyTZvGbgbFxH7, address:145.79.227.41:4094): protocol: read tcp 188.166.225.170:34288->145.79.227.41:4094: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:37.205179045 (331h34m22.667271528s) [noise=true , address=145.79.227.25:4646 , piece_id=CNB4NVHI657PMBJ5ZZGOXRAV2F6E26C5RHNC33FQLLBNEC74S65Q, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:1zA8TqySnmGc9bVQnNNMSZEPbv9Gf1Wa1BwYrKjc2ZGgtyEmrh, address:145.79.227.25:4646): protocol: write tcp 188.166.225.170:53664->145.79.227.25:4646: use of closed network connection; write tcp 188.166.225.170:53664->145.79.227.25:4646: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:53664->145.79.227.25:4646: use of closed network connection Oct 7 09:05:37.205212679 (331h34m22.667305182s) [noise=true , address=145.79.227.30:4786 , piece_id=7SWSXZX4US5UCDSEFVW3FYSAJ2VMX3U6NRAPF2P36F74DDGXDVJQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:125TWU7R4pkmrUxsNLhgTD475aLbjUJ7psA6drJtE7oDqZXYJ4Q, address:145.79.227.30:4786): protocol: write tcp 188.166.225.170:56580->145.79.227.30:4786: use of closed network connection; write tcp 188.166.225.170:56580->145.79.227.30:4786: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:56580->145.79.227.30:4786: use of closed network connection Oct 7 09:05:37.205352505 (331h34m22.667444989s) [noise=true , address=145.79.227.26:4679 , piece_id=225VDGLHHSPTDXYW2AZES72KQ2PXBI5WWRHL4PY4KEVKAPYCL4EA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:1VHQihNGcx6bTvwitH9PCSqyuKWuchdYhYzZXwSuoruQFByGoj, address:145.79.227.26:4679): protocol: read tcp 188.166.225.170:49732->145.79.227.26:4679: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:37.205396118 (331h34m22.667488601s) [noise=true , address=145.79.227.34:4931 , piece_id=YSG743WSKWN4CQPZINNXCLI2C7YKZ3KOETMPY2GK4CBAN54XXGPA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:12pCgoSfxjtXPw5jpbz9RJaxRiGoYPdPFdoMPtD19F1ZWY8UwPu, address:145.79.227.34:4931): protocol: write tcp 188.166.225.170:33504->145.79.227.34:4931: use of closed network connection; write tcp 188.166.225.170:33504->145.79.227.34:4931: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:33504->145.79.227.34:4931: use of closed network connection Oct 7 09:05:37.205691722 (331h34m22.667784205s) [noise=true , address=145.79.227.33:4900 , piece_id=P7YO24SJ5XVS7HOZFBRPGQ4KKNIT5OKIJVGCD3E3L5HPSFH7MBCA, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:1R8K65zsCspu7W2oAnvtpz7zEsQFkDG1e5BAp6faQYMQrkh4oQ, address:145.79.227.33:4900): protocol: read tcp 188.166.225.170:57608->145.79.227.33:4900: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:37.205901872 (331h34m22.667994345s) [noise=true , address=145.79.227.36:4552 , piece_id=DYBRNQP2EXHG6OPOSHUBWDZECJQEEABNITRASLEXH6KYG4QN5OGQ, seg_pos={0 0}, upload=291765]: Done uploading piece. err: ecclient: upload failed (node:1hoXgBFoGKKknrobbqfArxNhrsiUHKiiVZGWCnvxDSgBcae8gw, address:145.79.227.36:4552): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:05:37.205937039 (331h34m22.668029512s) [seg_pos={0 0}, upload=291765]: Done waiting for segment. successful: 11 optimal: 14 errs: <nil> Oct 7 09:05:37.452593759 (331h34m22.914686242s) [upload=291765]: Done uploading segments... Oct 7 09:05:37.452626602 (331h34m22.914719085s) [upload=291765]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 11) Duration: 1m0.249609311s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 7 out of 60 is failed (threshold=3) [...............................................X......XXXXXX] duration: 30.246146629s FAIL (2025-10-07 09:05:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Oct 7 09:04:41.357008775 (331h33m26.819101258s) [upload=291767]: Starting upload Oct 7 09:04:41.357096832 (331h33m26.819189305s) [upload=291767]: Done starting upload Oct 7 09:04:41.357323354 (331h33m26.819415837s) [upload=291767]: Uploading segments... Oct 7 09:04:41.357395301 (331h33m26.819487784s) [upload=291767]: Got next segment. Inline: false Oct 7 09:04:41.357434085 (331h33m26.819526558s) [upload=291767]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:04:41.357439124 (331h33m26.819531607s) [upload=291767]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:04:41.604834642 (331h33m27.066927115s) [seg_pos={0 0}, upload=291767]: Begin upload segment... Oct 7 09:04:41.604899536 (331h33m27.066992019s) [seg_pos={0 0}, upload=291767]: Done begin upload segment. Oct 7 09:04:41.604927389 (331h33m27.067019872s) [noise=true , address=45.91.116.7:4011 , piece_id=WSTNWQMI32DMDUBRWJKYCZSSUOXOAEH6GNTPFGJMSA55ZHZAYSUA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.604972084 (331h33m27.067064597s) [noise=true , address=145.79.227.17:4970 , piece_id=TWCHSM6LNEB36YCXVVFU5CCRRT6X2LNM2NVEZHSAPE4HKEEQTUGA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.605478008 (331h33m27.067570491s) [noise=true , address=77.90.31.11:4554 , piece_id=MEQYWRC7LNBIWV56HVBA5ZXSPATQLEDGJFTUB6DL2KHLEQCEIJAA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.605769242 (331h33m27.067861715s) [noise=true , address=145.79.227.13:4803 , piece_id=SOUHZV2RKO7VF6GRNOIP5TLQMTC5GAAWWXW3JH5JSWWI6LQVYMAQ, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.606108157 (331h33m27.06820064s) [noise=true , address=45.91.116.5:4298 , piece_id=PCYGLRILPEZB34LQ6Q54PHOF4X7XCKHGF27WCO4ZHNB4OW7F3GEQ, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.607209194 (331h33m27.069301677s) [noise=true , address=145.79.227.11:4732 , piece_id=HXTIXXSGIU5FCHG4ZYKJZWXOSB4EEQFDKW2PSUS5NREQWCUEZVQQ, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.607591522 (331h33m27.069684005s) [noise=true , address=77.90.31.10:4518 , piece_id=SEK2ODSAP4Z3KWJZYPHHRKFCKEWFGOWO7UBAKWWQF4OGHS4PL67Q, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.607967307 (331h33m27.07005982s) [noise=true , address=45.91.116.9:4262 , piece_id=2NU3D222GXD3NLZHZIB7MBFY5V3ZXKN4KYWASZFLG42VWPRSDRZA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.608367520 (331h33m27.070460003s) [noise=true , address=77.90.31.21:4879 , piece_id=MMK3JZRS4TA46RCENQESGLYFZJ75V2B6XKQYMYG4S4VZHQRSXHBA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.609838090 (331h33m27.071930573s) [noise=true , address=77.90.31.19:4841 , piece_id=TMHZPURFEQPBL5EXVXTYQV2BMMVP7GBQFONX2M4NTPAU5J64XM3A, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.609960373 (331h33m27.072052856s) [noise=true , address=145.79.227.15:4874 , piece_id=YDK5E6ZL5WKQE65FAWSMD22LWC75LZZ2DS6U3TGD3KWE6IZQOVXA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.611557134 (331h33m27.073649607s) [noise=true , address=145.79.227.16:4911 , piece_id=N3Q5NGY6IUTWMSRZ4O2QQOKWSKJ3FMWHBKB2BXBTQPNQ44YOVYUQ, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.613180284 (331h33m27.075272757s) [noise=true , address=145.79.227.17:4971 , piece_id=O7TQ4BCJT3PBXGQCIOPP2IWP563AB3QQN3CPQSNDMEJKZTDUBNDQ, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.614431717 (331h33m27.0765242s) [noise=true , address=77.90.31.16:4734 , piece_id=BGJFLGFW6PNB6ECODLI5IZCPSYGXR5ZMHIDKLPIP2NGLIH5LBQNA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.615582849 (331h33m27.077675332s) [noise=true , address=145.79.227.18:4007 , piece_id=5FHTCYN7XMVT6NEBYTZCWYFYQP3OONUOECKXDGS7H7JMN7E3LO5Q, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:41.616861784 (331h33m27.078954287s) [noise=true , address=45.91.116.12:4226 , piece_id=WEY3QQFGKQKBMJVDFUIUHVGAH3PZLCABAXVCDVDIAZOCHTC5BQAA, seg_pos={0 0}, upload=291767]: Uploading piece... Oct 7 09:04:43.364131397 (331h33m28.82622387s) [upload=291767]: Next returned nil segment Oct 7 09:04:43.364192292 (331h33m28.826284765s) [upload=291767]: Waiting for error group managing segments... Oct 7 09:04:43.695428792 (331h33m29.157521265s) [noise=true , address=77.90.31.10:4518 , piece_id=SEK2ODSAP4Z3KWJZYPHHRKFCKEWFGOWO7UBAKWWQF4OGHS4PL67Q, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.709655183 (331h33m29.171747656s) [noise=true , address=77.90.31.11:4554 , piece_id=MEQYWRC7LNBIWV56HVBA5ZXSPATQLEDGJFTUB6DL2KHLEQCEIJAA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.720252292 (331h33m29.182344775s) [noise=true , address=77.90.31.16:4734 , piece_id=BGJFLGFW6PNB6ECODLI5IZCPSYGXR5ZMHIDKLPIP2NGLIH5LBQNA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.757657981 (331h33m29.219750464s) [noise=true , address=45.91.116.5:4298 , piece_id=PCYGLRILPEZB34LQ6Q54PHOF4X7XCKHGF27WCO4ZHNB4OW7F3GEQ, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.770410405 (331h33m29.232502888s) [noise=true , address=45.91.116.12:4226 , piece_id=WEY3QQFGKQKBMJVDFUIUHVGAH3PZLCABAXVCDVDIAZOCHTC5BQAA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.776972561 (331h33m29.239065044s) [noise=true , address=45.91.116.7:4011 , piece_id=WSTNWQMI32DMDUBRWJKYCZSSUOXOAEH6GNTPFGJMSA55ZHZAYSUA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.794334272 (331h33m29.256426755s) [noise=true , address=45.91.116.9:4262 , piece_id=2NU3D222GXD3NLZHZIB7MBFY5V3ZXKN4KYWASZFLG42VWPRSDRZA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.851783478 (331h33m29.313875951s) [noise=true , address=77.90.31.19:4841 , piece_id=TMHZPURFEQPBL5EXVXTYQV2BMMVP7GBQFONX2M4NTPAU5J64XM3A, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:04:43.940954300 (331h33m29.403046783s) [noise=true , address=77.90.31.21:4879 , piece_id=MMK3JZRS4TA46RCENQESGLYFZJ75V2B6XKQYMYG4S4VZHQRSXHBA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: <nil> Oct 7 09:05:10.654786991 (331h33m56.116879464s) [noise=true , address=145.79.227.16:4911 , piece_id=N3Q5NGY6IUTWMSRZ4O2QQOKWSKJ3FMWHBKB2BXBTQPNQ44YOVYUQ, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 188.166.225.170:40636->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:10.654957386 (331h33m56.117049859s) [noise=true , address=145.79.227.13:4803 , piece_id=SOUHZV2RKO7VF6GRNOIP5TLQMTC5GAAWWXW3JH5JSWWI6LQVYMAQ, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: read tcp 188.166.225.170:48728->145.79.227.13:4803: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:10.654978706 (331h33m56.11707119s) [noise=true , address=145.79.227.17:4970 , piece_id=TWCHSM6LNEB36YCXVVFU5CCRRT6X2LNM2NVEZHSAPE4HKEEQTUGA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:55698->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:10.655099476 (331h33m56.117191969s) [noise=true , address=145.79.227.15:4874 , piece_id=YDK5E6ZL5WKQE65FAWSMD22LWC75LZZ2DS6U3TGD3KWE6IZQOVXA, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: rpc: dial tcp 145.79.227.15:4874: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:10.655136797 (331h33m56.11722928s) [noise=true , address=145.79.227.11:4732 , piece_id=HXTIXXSGIU5FCHG4ZYKJZWXOSB4EEQFDKW2PSUS5NREQWCUEZVQQ, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: read tcp 188.166.225.170:57398->145.79.227.11:4732: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:05:10.655349532 (331h33m56.117442025s) [noise=true , address=145.79.227.17:4971 , piece_id=O7TQ4BCJT3PBXGQCIOPP2IWP563AB3QQN3CPQSNDMEJKZTDUBNDQ, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:12Ut4J9xtQTDXFFmCSv9rk6sBSjNpEdnyxo6cZWMXcpAvuQtebo, address:145.79.227.17:4971): protocol: read tcp 188.166.225.170:57274->145.79.227.17:4971: use of closed network connection; context deadline exceeded Oct 7 09:05:10.655450344 (331h33m56.117542837s) [noise=true , address=145.79.227.18:4007 , piece_id=5FHTCYN7XMVT6NEBYTZCWYFYQP3OONUOECKXDGS7H7JMN7E3LO5Q, seg_pos={0 0}, upload=291767]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:05:10.655489819 (331h33m56.117582292s) [seg_pos={0 0}, upload=291767]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 09:05:10.898677085 (331h33m56.360769578s) [upload=291767]: Done uploading segments... Oct 7 09:05:10.898714416 (331h33m56.360806909s) [upload=291767]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.246165175s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 6 out of 60 is failed (threshold=3) [................................................X......XXXXX] duration: 30.257015541s FAIL (2025-10-07 09:04:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Oct 7 09:04:10.863674079 (331h32m56.325766562s) [upload=291759]: Starting upload Oct 7 09:04:10.863782405 (331h32m56.325874898s) [upload=291759]: Done starting upload Oct 7 09:04:10.863915238 (331h32m56.326007721s) [upload=291759]: Uploading segments... Oct 7 09:04:10.864064743 (331h32m56.326157226s) [upload=291759]: Got next segment. Inline: false Oct 7 09:04:10.864117824 (331h32m56.326210307s) [upload=291759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:04:10.864124607 (331h32m56.32621709s) [upload=291759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:04:11.193598700 (331h32m56.655691173s) [seg_pos={0 0}, upload=291759]: Begin upload segment... Oct 7 09:04:11.193659096 (331h32m56.655751579s) [seg_pos={0 0}, upload=291759]: Done begin upload segment. Oct 7 09:04:11.193685285 (331h32m56.655777768s) [noise=true , address=145.79.227.13:4802 , piece_id=OVFI57HODWGQ4WAS6NVIRSZC4KNJ4OQVVGLSZA6NHDAELME3BY3A, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.193693361 (331h32m56.655785844s) [noise=true , address=45.91.116.7:4009 , piece_id=VFIMF6R2NYLRSENIOMPEKDFADSA3HPVIOUWENXB7XZTQWIIJT4BQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.194190117 (331h32m56.6562826s) [noise=true , address=145.79.227.11:4729 , piece_id=UG7XZ3TF32UFDA4QZRR5AAN3GXDM3DS6G2JYTRKEAVJWM5G6WPHA, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.194880742 (331h32m56.656973215s) [noise=true , address=77.90.31.11:4555 , piece_id=ZLERP33RO7OX2BZWJAXGT5SDM67KAAKYTA4EQPJWHH73DOOT25IQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.195357901 (331h32m56.657450384s) [noise=true , address=77.90.31.21:4878 , piece_id=VHJ327XAPWSGJR4N5CFZ4ACQTRQYEBXIMHBI5FVEHV6Y73CXF3OQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.195664464 (331h32m56.657756947s) [noise=true , address=45.91.116.13:4153 , piece_id=3RAGCVSJFPPIYKX7PRUIZ22A5BLJVAELXYWRWPEDGSYAC2HQXLMQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.196038376 (331h32m56.658130859s) [noise=true , address=45.91.116.8:4048 , piece_id=ZQVQ5VLMT4MHPS4CD2I2MUCISSJ4III4XY6YXOMKOJSWVFUNH7QA, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.196438328 (331h32m56.658530811s) [noise=true , address=77.90.31.15:4697 , piece_id=P3KEYDRTO6NKQFQZTGGZWM6ZJAUPXFGRFD5PRDQGVRPOKRVV2JMQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.198046540 (331h32m56.660139023s) [noise=true , address=77.90.31.19:4842 , piece_id=63STFMTJH5GEGOYPBP2TI62JTW4P3XYGIXPYH47RMJCE2UGFFU3Q, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.198234438 (331h32m56.660326911s) [noise=true , address=145.79.227.2:4115 , piece_id=RVSAG3PYDO2TWEAQWLFA6KELKUL3SUZOGIUCHAV3IH3OSCLE5HZA, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.199728192 (331h32m56.661820675s) [noise=true , address=45.91.116.14:4189 , piece_id=CDSF3C3IUMGTD6Z76LP6XX5742MYEA4ROUNEJFQSSKOKVGNJGTDQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.199754021 (331h32m56.661846504s) [noise=true , address=145.79.227.14:4838 , piece_id=PNJK37FIFRJO5TH5E5WJAUBKHXTBPMTYT4AOLQDBO2LNFLVGWQ3A, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.201467996 (331h32m56.663560469s) [noise=true , address=145.79.227.5:4223 , piece_id=5CDS5V7VVNHKLDFGS6KAZONXPFIEAP2AXJ3UB74FT4HGKKR73XRA, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.209099988 (331h32m56.671192491s) [noise=true , address=145.79.227.6:4257 , piece_id=VPC2VVBTS6VG6YXU75IBGXEAMZCJMP2QKFKMGOU5KFXM2HIUGLVQ, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.211638392 (331h32m56.673730875s) [noise=true , address=77.90.31.13:4626 , piece_id=U6RYVHRY5CE2ZPIOSA7QEOIFVH2EYNQUYU5I424SOIMNZVUQYGJA, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:11.212952875 (331h32m56.675045348s) [noise=true , address=77.90.31.16:4734 , piece_id=OGBT4FI4J5YIPS4FBYFAPMH2S7FXYX4PADIQBMNM3DCAXIACAIPA, seg_pos={0 0}, upload=291759]: Uploading piece... Oct 7 09:04:12.908205499 (331h32m58.370297983s) [upload=291759]: Next returned nil segment Oct 7 09:04:12.908227080 (331h32m58.370319563s) [upload=291759]: Waiting for error group managing segments... Oct 7 09:04:13.317586645 (331h32m58.779679128s) [noise=true , address=77.90.31.16:4734 , piece_id=OGBT4FI4J5YIPS4FBYFAPMH2S7FXYX4PADIQBMNM3DCAXIACAIPA, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.335895048 (331h32m58.797987531s) [noise=true , address=45.91.116.13:4153 , piece_id=3RAGCVSJFPPIYKX7PRUIZ22A5BLJVAELXYWRWPEDGSYAC2HQXLMQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.343318483 (331h32m58.805410966s) [noise=true , address=45.91.116.7:4009 , piece_id=VFIMF6R2NYLRSENIOMPEKDFADSA3HPVIOUWENXB7XZTQWIIJT4BQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.350444063 (331h32m58.812536536s) [noise=true , address=45.91.116.8:4048 , piece_id=ZQVQ5VLMT4MHPS4CD2I2MUCISSJ4III4XY6YXOMKOJSWVFUNH7QA, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.350532441 (331h32m58.812624924s) [noise=true , address=77.90.31.11:4555 , piece_id=ZLERP33RO7OX2BZWJAXGT5SDM67KAAKYTA4EQPJWHH73DOOT25IQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.415252733 (331h32m58.877345206s) [noise=true , address=77.90.31.21:4878 , piece_id=VHJ327XAPWSGJR4N5CFZ4ACQTRQYEBXIMHBI5FVEHV6Y73CXF3OQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.432088632 (331h32m58.894181105s) [noise=true , address=45.91.116.14:4189 , piece_id=CDSF3C3IUMGTD6Z76LP6XX5742MYEA4ROUNEJFQSSKOKVGNJGTDQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.454047317 (331h32m58.91613979s) [noise=true , address=77.90.31.15:4697 , piece_id=P3KEYDRTO6NKQFQZTGGZWM6ZJAUPXFGRFD5PRDQGVRPOKRVV2JMQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.505145550 (331h32m58.967238033s) [noise=true , address=77.90.31.13:4626 , piece_id=U6RYVHRY5CE2ZPIOSA7QEOIFVH2EYNQUYU5I424SOIMNZVUQYGJA, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:13.525516040 (331h32m58.987608523s) [noise=true , address=77.90.31.19:4842 , piece_id=63STFMTJH5GEGOYPBP2TI62JTW4P3XYGIXPYH47RMJCE2UGFFU3Q, seg_pos={0 0}, upload=291759]: Done uploading piece. err: <nil> Oct 7 09:04:40.154805824 (331h33m25.616898297s) [noise=true , address=145.79.227.2:4115 , piece_id=RVSAG3PYDO2TWEAQWLFA6KELKUL3SUZOGIUCHAV3IH3OSCLE5HZA, seg_pos={0 0}, upload=291759]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:46766->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:46766->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46766->145.79.227.2:4115: use of closed network connection Oct 7 09:04:40.155083913 (331h33m25.617176396s) [noise=true , address=145.79.227.14:4838 , piece_id=PNJK37FIFRJO5TH5E5WJAUBKHXTBPMTYT4AOLQDBO2LNFLVGWQ3A, seg_pos={0 0}, upload=291759]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: rpc: dial tcp 145.79.227.14:4838: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:40.155249789 (331h33m25.617342272s) [noise=true , address=145.79.227.5:4223 , piece_id=5CDS5V7VVNHKLDFGS6KAZONXPFIEAP2AXJ3UB74FT4HGKKR73XRA, seg_pos={0 0}, upload=291759]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: read tcp 188.166.225.170:52666->145.79.227.5:4223: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:40.155308481 (331h33m25.617400964s) [noise=true , address=145.79.227.11:4729 , piece_id=UG7XZ3TF32UFDA4QZRR5AAN3GXDM3DS6G2JYTRKEAVJWM5G6WPHA, seg_pos={0 0}, upload=291759]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: read tcp 188.166.225.170:32992->145.79.227.11:4729: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:40.155483755 (331h33m25.617576238s) [noise=true , address=145.79.227.6:4257 , piece_id=VPC2VVBTS6VG6YXU75IBGXEAMZCJMP2QKFKMGOU5KFXM2HIUGLVQ, seg_pos={0 0}, upload=291759]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:145.79.227.6:4257): protocol: read tcp 188.166.225.170:60238->145.79.227.6:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:40.155545032 (331h33m25.617637515s) [noise=true , address=145.79.227.13:4802 , piece_id=OVFI57HODWGQ4WAS6NVIRSZC4KNJ4OQVVGLSZA6NHDAELME3BY3A, seg_pos={0 0}, upload=291759]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: read tcp 188.166.225.170:57162->145.79.227.13:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:40.155576933 (331h33m25.617669416s) [seg_pos={0 0}, upload=291759]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Oct 7 09:04:40.410571160 (331h33m25.872663643s) [upload=291759]: Done uploading segments... Oct 7 09:04:40.410608571 (331h33m25.872701044s) [upload=291759]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 30.257030469s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 1m0.261031654s FAIL (2025-10-07 09:04:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 12) ======LOGS====== Z4Q3QZUX3FEVA5JY2RUNWT5UW7KFA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.660724621 (331h32m23.122817094s) [noise=true , address=77.90.31.10:4530 , piece_id=5QZCLO4W6H7ZOY6IUB7OXMVFP3OUPCDPEP6AFNCNIT3P42G7UAGA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.660806196 (331h32m23.122898669s) [noise=true , address=145.79.227.41:4101 , piece_id=3SKMWMTXHNHB2C2EHSAHF5CRZGKKKT7KEJXSPK4FAIO4TAXIV6SQ, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.661061402 (331h32m23.123153876s) [noise=true , address=45.91.116.7:4015 , piece_id=CERCFIPS42ILF6HOIQ4PZ2NXVIT5LYKYLBL5AGHTV72RQO45NGXA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.661317089 (331h32m23.123409572s) [noise=true , address=145.79.227.49:4345 , piece_id=FH3PF744OYZZK5Q47DS4NMUVS6PKE5GLKUOTWZKO6OECW2TKDXGA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.661578417 (331h32m23.1236709s) [noise=true , address=77.90.31.19:4868 , piece_id=LJBR4X5RPA6N2OMDZ7WXKKY6AWPYCCPPY6TITOKF3SBC6CBR3IMA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.662492808 (331h32m23.124585291s) [noise=true , address=77.90.31.16:4740 , piece_id=SNIVLP3XT4DYGEWWWWVHWG5BIRYTVOQ3UYPH2HDHDYNS6JNHXIUQ, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.662780305 (331h32m23.124872788s) [noise=true , address=77.90.31.17:4780 , piece_id=M2ZJHUKML6IPEYQKFVQU2LS23RGTSYCK4JQ46CNXGRYZLJOC3BRQ, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.663118149 (331h32m23.125210622s) [noise=true , address=145.79.227.34:4910 , piece_id=V746SJ5I3PSY5YIEXLFEANGKULUJ4P5U3CXDL7XS3SVAFREUOSIA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.663492622 (331h32m23.125585105s) [noise=true , address=45.91.116.12:4242 , piece_id=PGAKHLWZMZPHREWZBIBET36P7AVOSE4AFZPHRNUTP3CEO3U2XW4A, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.663885229 (331h32m23.125977712s) [noise=true , address=64.49.13.138:4482 , piece_id=OHM7PJ5LIK32R64B5DZO3MLXAQKWJDHVEPDLNF2WWRSIHHOMTN7A, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.664120237 (331h32m23.12621271s) [noise=true , address=45.91.116.11:4143 , piece_id=Y27XJHM7FC3FQRY3AUDQV5SI23KPIE4GI7RO2LKT7GMOIBNDV5OQ, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.667215862 (331h32m23.129308345s) [noise=true , address=77.90.31.12:4594 , piece_id=KEX7PTXJ764PQMQ52H3U3HSGM6A6QXQKOLKPS7PYMES574S5BZUA, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.668768248 (331h32m23.130860731s) [noise=true , address=77.90.31.11:4565 , piece_id=MLAFITWPJXKO2VRJBMD4F6X4DUGS25SHX46UT74TCLFRWQNVRVNQ, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.669878441 (331h32m23.131970914s) [noise=true , address=145.79.227.52:4450 , piece_id=YWM4UMSWJ4CBMIMSAUHHGN5DFXKOS4ZZVUBJDOF2TCCJJC2XUIXQ, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.671018402 (331h32m23.133110885s) [noise=true , address=45.91.116.10:4088 , piece_id=6UUQLHDI4MEVPF55EF6W7OIM6UGMPSZWXTHXYJMUGGBYEL72ND2Q, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.672160397 (331h32m23.13425287s) [noise=true , address=145.79.227.24:4602 , piece_id=RMQJKWPJXWDZBFSNVRCPV6BCKQVLZNWAEYUKINNKNSOR4MFASQ2Q, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:37.673281011 (331h32m23.135373494s) [noise=true , address=145.79.227.32:4843 , piece_id=QQDOBL66BF4HKJCCBOJY5UBARMHWBDEHETFFWOYV4MKAUM7MZP2A, seg_pos={0 0}, upload=291750]: Uploading piece... Oct 7 09:03:39.026280033 (331h32m24.488372516s) [upload=291750]: Next returned nil segment Oct 7 09:03:39.026298848 (331h32m24.488391321s) [upload=291750]: Waiting for error group managing segments... Oct 7 09:03:39.392911263 (331h32m24.855003736s) [noise=true , address=64.49.13.138:4482 , piece_id=OHM7PJ5LIK32R64B5DZO3MLXAQKWJDHVEPDLNF2WWRSIHHOMTN7A, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.612902276 (331h32m25.074994759s) [noise=true , address=77.90.31.19:4868 , piece_id=LJBR4X5RPA6N2OMDZ7WXKKY6AWPYCCPPY6TITOKF3SBC6CBR3IMA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.619149763 (331h32m25.081242246s) [noise=true , address=45.91.116.12:4242 , piece_id=PGAKHLWZMZPHREWZBIBET36P7AVOSE4AFZPHRNUTP3CEO3U2XW4A, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.626910100 (331h32m25.089002573s) [noise=true , address=45.91.116.10:4088 , piece_id=6UUQLHDI4MEVPF55EF6W7OIM6UGMPSZWXTHXYJMUGGBYEL72ND2Q, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.671072335 (331h32m25.133164828s) [noise=true , address=77.90.31.17:4780 , piece_id=M2ZJHUKML6IPEYQKFVQU2LS23RGTSYCK4JQ46CNXGRYZLJOC3BRQ, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.677394484 (331h32m25.139486967s) [noise=true , address=77.90.31.13:4635 , piece_id=2SQIE7UPHJGMF5GWMY75DAEZ4Q3QZUX3FEVA5JY2RUNWT5UW7KFA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.678290290 (331h32m25.140382773s) [noise=true , address=45.91.116.11:4143 , piece_id=Y27XJHM7FC3FQRY3AUDQV5SI23KPIE4GI7RO2LKT7GMOIBNDV5OQ, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.695029756 (331h32m25.157122229s) [noise=true , address=77.90.31.11:4565 , piece_id=MLAFITWPJXKO2VRJBMD4F6X4DUGS25SHX46UT74TCLFRWQNVRVNQ, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.719860829 (331h32m25.181953302s) [noise=true , address=77.90.31.12:4594 , piece_id=KEX7PTXJ764PQMQ52H3U3HSGM6A6QXQKOLKPS7PYMES574S5BZUA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.744859072 (331h32m25.206951555s) [noise=true , address=45.91.116.7:4015 , piece_id=CERCFIPS42ILF6HOIQ4PZ2NXVIT5LYKYLBL5AGHTV72RQO45NGXA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.756046254 (331h32m25.218138727s) [noise=true , address=77.90.31.16:4740 , piece_id=SNIVLP3XT4DYGEWWWWVHWG5BIRYTVOQ3UYPH2HDHDYNS6JNHXIUQ, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:03:39.786367693 (331h32m25.248460176s) [noise=true , address=77.90.31.10:4530 , piece_id=5QZCLO4W6H7ZOY6IUB7OXMVFP3OUPCDPEP6AFNCNIT3P42G7UAGA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: <nil> Oct 7 09:04:36.689113450 (331h33m22.151205934s) [noise=true , address=145.79.227.49:4345 , piece_id=FH3PF744OYZZK5Q47DS4NMUVS6PKE5GLKUOTWZKO6OECW2TKDXGA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: ecclient: upload failed (node:1KSTyRJ62wcMHx8K7uwqp5e8L8yhxH7V3YtBHnYu6QJ3K3bXpQ, address:145.79.227.49:4345): protocol: write tcp 188.166.225.170:34604->145.79.227.49:4345: use of closed network connection; write tcp 188.166.225.170:34604->145.79.227.49:4345: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:34604->145.79.227.49:4345: use of closed network connection Oct 7 09:04:36.689270158 (331h33m22.151362641s) [noise=true , address=145.79.227.41:4101 , piece_id=3SKMWMTXHNHB2C2EHSAHF5CRZGKKKT7KEJXSPK4FAIO4TAXIV6SQ, seg_pos={0 0}, upload=291750]: Done uploading piece. err: ecclient: upload failed (node:12WWXeBjFZR9PzciNfs35fFcx87N9GwDpMsPHnksFDugtaBGWp4, address:145.79.227.41:4101): protocol: write tcp 188.166.225.170:50638->145.79.227.41:4101: use of closed network connection; write tcp 188.166.225.170:50638->145.79.227.41:4101: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:50638->145.79.227.41:4101: use of closed network connection Oct 7 09:04:36.689445433 (331h33m22.151537916s) [noise=true , address=145.79.227.24:4602 , piece_id=RMQJKWPJXWDZBFSNVRCPV6BCKQVLZNWAEYUKINNKNSOR4MFASQ2Q, seg_pos={0 0}, upload=291750]: Done uploading piece. err: ecclient: upload failed (node:12oPZgRG4QqgnQLo1fgyGYoX1zd34wsd5n7p7L3mRbPD8B4JUuB, address:145.79.227.24:4602): protocol: read tcp 188.166.225.170:41616->145.79.227.24:4602: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:36.689568878 (331h33m22.151661361s) [noise=true , address=145.79.227.32:4843 , piece_id=QQDOBL66BF4HKJCCBOJY5UBARMHWBDEHETFFWOYV4MKAUM7MZP2A, seg_pos={0 0}, upload=291750]: Done uploading piece. err: ecclient: upload failed (node:1bBR6KYQ1sSfvunbVsY8t12xrN6uk2kjq8ccDcFiopYSX72tet, address:145.79.227.32:4843): protocol: read tcp 188.166.225.170:48764->145.79.227.32:4843: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:36.689739332 (331h33m22.151831825s) [noise=true , address=145.79.227.52:4450 , piece_id=YWM4UMSWJ4CBMIMSAUHHGN5DFXKOS4ZZVUBJDOF2TCCJJC2XUIXQ, seg_pos={0 0}, upload=291750]: Done uploading piece. err: ecclient: upload failed (node:12XE5kZtVU6FeUR5NF2mhwypKiJahTUyt4qtRbo3PUJp6T5sdms, address:145.79.227.52:4450): protocol: read tcp 188.166.225.170:32890->145.79.227.52:4450: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:36.690037340 (331h33m22.152129823s) [noise=true , address=145.79.227.34:4910 , piece_id=V746SJ5I3PSY5YIEXLFEANGKULUJ4P5U3CXDL7XS3SVAFREUOSIA, seg_pos={0 0}, upload=291750]: Done uploading piece. err: ecclient: upload failed (node:12bRYzNggkq7QWFTP9fB94dTrMMX8AYvKMt2zjmJBzT1VwqyPVZ, address:145.79.227.34:4910): protocol: read tcp 188.166.225.170:41270->145.79.227.34:4910: use of closed network connection; context deadline exceeded Oct 7 09:04:36.690091713 (331h33m22.152184196s) [seg_pos={0 0}, upload=291750]: Done waiting for segment. successful: 12 optimal: 14 errs: <nil> Oct 7 09:04:36.947851958 (331h33m22.409944451s) [upload=291750]: Done uploading segments... Oct 7 09:04:36.947892905 (331h33m22.409985388s) [upload=291750]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 12) Duration: 1m0.261050059s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 5 out of 60 is failed (threshold=3) [.................................................X......XXXX] duration: 30.243427685s FAIL (2025-10-07 09:04:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== Oct 7 09:03:40.365080484 (331h32m25.827172967s) [upload=291751]: Starting upload Oct 7 09:03:40.365170075 (331h32m25.827262558s) [upload=291751]: Done starting upload Oct 7 09:03:40.365297117 (331h32m25.82738962s) [upload=291751]: Uploading segments... Oct 7 09:03:40.365439007 (331h32m25.82753148s) [upload=291751]: Got next segment. Inline: false Oct 7 09:03:40.365487459 (331h32m25.827579952s) [upload=291751]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:03:40.365493009 (331h32m25.827585502s) [upload=291751]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:03:40.693728014 (331h32m26.155820497s) [seg_pos={0 0}, upload=291751]: Begin upload segment... Oct 7 09:03:40.693825500 (331h32m26.155917983s) [seg_pos={0 0}, upload=291751]: Done begin upload segment. Oct 7 09:03:40.693863592 (331h32m26.155956086s) [noise=true , address=77.90.31.21:4879 , piece_id=3FFMY3SAPFYZU7K6HSLHFRMWOHL5QH4XYJWVJ2FL35LZJJANH5ZA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.693935710 (331h32m26.156028183s) [noise=true , address=45.91.116.12:4227 , piece_id=Z7RQSBEPSMZSCSF47RLJU4WTBVKWFZNK6CGJQO22DMRBX2KN6GWA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.694555781 (331h32m26.156648254s) [noise=true , address=145.79.227.13:4802 , piece_id=M4GW4JPZMLAQEAIMZY3QNAMQUQ3DH2BPY4SS2HKF2GCKTTQ46FOQ, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.694961804 (331h32m26.157054287s) [noise=true , address=145.79.227.14:4837 , piece_id=BOQDYHLSQNXDLGWBCTDB3VLBNY3KSXFYP4CYOGUXOX56QQCDYWXQ, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.695317702 (331h32m26.157410185s) [noise=true , address=45.91.116.6:4973 , piece_id=57QY4KZCJKBUVNO55ZMW2MAAUJQYYAQKPIDEBRVWEMUTOQHJE3AA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.695616160 (331h32m26.157708633s) [noise=true , address=145.79.227.4:4185 , piece_id=EF55CTU4IYNTW3NAOJGAZLWLK7AOP3LEQWUCJYKUL3BKTFAGSS5A, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.695983499 (331h32m26.158075972s) [noise=true , address=145.79.227.17:4969 , piece_id=5W4ZCKYU6FBRWXJDQSTFYQFPCBSVYEFUWVXKRJ2DGLJ5EO2K5JKQ, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.696292488 (331h32m26.158384971s) [noise=true , address=45.91.116.4:4939 , piece_id=ESOSETNFGPAGBA4NU7GSDHMI3B2RVSJ3JD3ZTFVITUAZVFZAF3MA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.696848957 (331h32m26.15894144s) [noise=true , address=45.91.116.10:4081 , piece_id=Y4IEVMGJAIUBSTFJDKIK63L546NRNX2BJ6O5YOPSK3YRMFJ5FUXA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.699354208 (331h32m26.161446681s) [noise=true , address=145.79.227.2:4114 , piece_id=2ZZ5JH77VBJOYKEJJVWO23L33X4B43POPG5DDTPRJJJAEKYN2HZA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.703191786 (331h32m26.165284269s) [noise=true , address=45.91.116.5:4298 , piece_id=T5KSFMM7IXJH4VTGA256S7R5T7FWPYX5YFC6JO5GV5VLZOF6RYZA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.704737619 (331h32m26.166830092s) [noise=true , address=145.79.227.6:4258 , piece_id=LU5CSYVV7RI467FPLN7ZJOQAZAL2BFYHNZ6CAZWXW6WCPVPUPIFQ, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.706906600 (331h32m26.168999083s) [noise=true , address=77.90.31.19:4843 , piece_id=ZWTUIFHJR6XIYSF3LQQEIRFGGRACCB3YQSR4AP2N5SHDIZY6KUTA, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.708961413 (331h32m26.171053896s) [noise=true , address=145.79.227.17:4970 , piece_id=FE5HCMUWE6QAJK7TERCYZUV7JGNME6U63OMUJS5B2TLY6QCPOIEQ, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.710715512 (331h32m26.172807985s) [noise=true , address=45.91.116.9:4262 , piece_id=BI3KJEQXTWJC4B742IBPBLUD6DYNLV7PA7IV6POJQH6XRDSVS5UQ, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:40.713991570 (331h32m26.176084053s) [noise=true , address=77.90.31.10:4518 , piece_id=RMOK5ANAEBMVXW3YLLNLYTDWVZIWNMRGZFQWBJSAQOJKRUPRZM4A, seg_pos={0 0}, upload=291751]: Uploading piece... Oct 7 09:03:42.386243742 (331h32m27.848336215s) [upload=291751]: Next returned nil segment Oct 7 09:03:42.386265333 (331h32m27.848357806s) [upload=291751]: Waiting for error group managing segments... Oct 7 09:03:42.770582330 (331h32m28.232674803s) [noise=true , address=45.91.116.10:4081 , piece_id=Y4IEVMGJAIUBSTFJDKIK63L546NRNX2BJ6O5YOPSK3YRMFJ5FUXA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.791276877 (331h32m28.25336936s) [noise=true , address=77.90.31.21:4879 , piece_id=3FFMY3SAPFYZU7K6HSLHFRMWOHL5QH4XYJWVJ2FL35LZJJANH5ZA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.814902306 (331h32m28.276994789s) [noise=true , address=45.91.116.4:4939 , piece_id=ESOSETNFGPAGBA4NU7GSDHMI3B2RVSJ3JD3ZTFVITUAZVFZAF3MA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.834220131 (331h32m28.296312614s) [noise=true , address=45.91.116.12:4227 , piece_id=Z7RQSBEPSMZSCSF47RLJU4WTBVKWFZNK6CGJQO22DMRBX2KN6GWA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.840905301 (331h32m28.302997784s) [noise=true , address=45.91.116.6:4973 , piece_id=57QY4KZCJKBUVNO55ZMW2MAAUJQYYAQKPIDEBRVWEMUTOQHJE3AA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.859021519 (331h32m28.321113992s) [noise=true , address=45.91.116.9:4262 , piece_id=BI3KJEQXTWJC4B742IBPBLUD6DYNLV7PA7IV6POJQH6XRDSVS5UQ, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.871830821 (331h32m28.333923304s) [noise=true , address=45.91.116.5:4298 , piece_id=T5KSFMM7IXJH4VTGA256S7R5T7FWPYX5YFC6JO5GV5VLZOF6RYZA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.897131810 (331h32m28.359224293s) [noise=true , address=77.90.31.10:4518 , piece_id=RMOK5ANAEBMVXW3YLLNLYTDWVZIWNMRGZFQWBJSAQOJKRUPRZM4A, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:03:42.931896820 (331h32m28.393989293s) [noise=true , address=77.90.31.19:4843 , piece_id=ZWTUIFHJR6XIYSF3LQQEIRFGGRACCB3YQSR4AP2N5SHDIZY6KUTA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: <nil> Oct 7 09:04:09.671645440 (331h32m55.133737923s) [noise=true , address=145.79.227.14:4837 , piece_id=BOQDYHLSQNXDLGWBCTDB3VLBNY3KSXFYP4CYOGUXOX56QQCDYWXQ, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: read tcp 188.166.225.170:33886->145.79.227.14:4837: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:09.671753315 (331h32m55.133845808s) [noise=true , address=145.79.227.17:4969 , piece_id=5W4ZCKYU6FBRWXJDQSTFYQFPCBSVYEFUWVXKRJ2DGLJ5EO2K5JKQ, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:145.79.227.17:4969): protocol: read tcp 188.166.225.170:59976->145.79.227.17:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:09.671764357 (331h32m55.13385683s) [noise=true , address=145.79.227.2:4114 , piece_id=2ZZ5JH77VBJOYKEJJVWO23L33X4B43POPG5DDTPRJJJAEKYN2HZA, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:1rbW6vFBsSXTwat2th5XhcKj4Jmnsupc89u7KZ7zNh84JmE4JN, address:145.79.227.2:4114): protocol: read tcp 188.166.225.170:36022->145.79.227.2:4114: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:09.671906768 (331h32m55.133999251s) [noise=true , address=145.79.227.4:4185 , piece_id=EF55CTU4IYNTW3NAOJGAZLWLK7AOP3LEQWUCJYKUL3BKTFAGSS5A, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: read tcp 188.166.225.170:54612->145.79.227.4:4185: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:09.671977723 (331h32m55.134070206s) [noise=true , address=145.79.227.6:4258 , piece_id=LU5CSYVV7RI467FPLN7ZJOQAZAL2BFYHNZ6CAZWXW6WCPVPUPIFQ, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: read tcp 188.166.225.170:57234->145.79.227.6:4258: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:09.671993262 (331h32m55.134085735s) [noise=true , address=145.79.227.17:4970 , piece_id=FE5HCMUWE6QAJK7TERCYZUV7JGNME6U63OMUJS5B2TLY6QCPOIEQ, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:1Fyc4xk3wSbE57sX7JZG3mex5Jwdqb7aey6PXSbCKHzkD7KUVe, address:145.79.227.17:4970): protocol: read tcp 188.166.225.170:54670->145.79.227.17:4970: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:04:09.672141405 (331h32m55.134233888s) [noise=true , address=145.79.227.13:4802 , piece_id=M4GW4JPZMLAQEAIMZY3QNAMQUQ3DH2BPY4SS2HKF2GCKTTQ46FOQ, seg_pos={0 0}, upload=291751]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: context deadline exceeded; context deadline exceeded Oct 7 09:04:09.672178966 (331h32m55.134271449s) [seg_pos={0 0}, upload=291751]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 09:04:09.913488374 (331h32m55.375580867s) [upload=291751]: Done uploading segments... Oct 7 09:04:09.913522389 (331h32m55.375614872s) [upload=291751]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.243446792s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................X......XXX] duration: 30.254764995s FAIL (2025-10-07 09:03:39 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 7) ======LOGS====== 31h31m55.598902755s) [noise=true , address=77.90.31.21:4878 , piece_id=BA4Z5DSVFKXTIONVXEXYNCJNBKQVBZCREX5HEE4JUO2KGDI6APPQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.137226745 (331h31m55.599319229s) [noise=true , address=145.79.227.16:4909 , piece_id=BNG6I57HLTODTJQLEQ47VJMEMA3TECGDTZKBER3E5HPI3MGBO6OQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.137597492 (331h31m55.599689975s) [noise=true , address=145.79.227.5:4221 , piece_id=QD2QUXRE5JTWU3MCKMKBZ5C2O64BCSIRIVMHDTY7Z3I3G37SY2TQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.138197514 (331h31m55.600289997s) [noise=true , address=77.90.31.13:4626 , piece_id=ASQNXHKONSPUF75LMTT7FGQKVAUR7XGEBSITVMISEKEWFCRLJF6Q, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.138527773 (331h31m55.600620246s) [noise=true , address=145.79.227.15:4874 , piece_id=OYALXEG4UMIPI2R3HSPMVNSHSVEKXOJWDCRWX7CBHG5WRYUV6HKA, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.138938506 (331h31m55.601030989s) [noise=true , address=77.90.31.12:4589 , piece_id=OL63W5YTVYOWO4HT7HDYYQIQKVYOXNAWTJPR4XIDDFO7325PHN7A, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.139272932 (331h31m55.601365415s) [noise=true , address=145.79.227.16:4911 , piece_id=3SRKS4TUFZQTDH7UNAUYTC2AFSP2LBP2TPTX6T2CUZQTIUCYGCWQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.139623620 (331h31m55.601716093s) [noise=true , address=45.91.116.13:4153 , piece_id=VDD3KULKWJOZU2MHFA5PQISI2JLIRYEKI3OTL5VGUXL3GTO7NNMA, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.139957396 (331h31m55.602049879s) [noise=true , address=145.79.227.19:4079 , piece_id=2GQDXR54ZP4YWPLMW3FARMLWDGIRDGBG7O5LKRHHZ5H2T4VCCXEQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.140245774 (331h31m55.602338248s) [noise=true , address=145.79.227.18:4006 , piece_id=SUO6MBOKAC3IBP237Z4NBXPFDUI37NVE6GRZDVBKNL5I5SGMWCCQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.141282669 (331h31m55.603375152s) [noise=true , address=77.90.31.14:4662 , piece_id=GBVAUJTRKS24TNGLQ6Q7UMS6ZHP354AO7R3QAC6C5G46JLO24RRA, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.142017208 (331h31m55.604109691s) [noise=true , address=77.90.31.18:4807 , piece_id=DBD3RWNTQRDBLRUJZCFOKTFMYDCZGX4CCOHJIBWYTHFAUWIMY54A, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.142445994 (331h31m55.604538467s) [noise=true , address=145.79.227.12:4767 , piece_id=ITXO2MQ5PASVTP2XDDCRFDKCRORVQY7M3JZCUPNADVS7A27X7KJQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.142813674 (331h31m55.604906147s) [noise=true , address=145.79.227.3:4151 , piece_id=2A7FM5BJTLRZCEARQEQEGQC73OJ7U5N4I5H3Q5REZO46R5SSQLYQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:10.143221241 (331h31m55.605313724s) [noise=true , address=45.91.116.4:4937 , piece_id=6KC5OU3GDXYX27JMU5XG4TRSE6MF3ITQ4FJGUZIRXOEHRYNVDPUQ, seg_pos={0 0}, upload=291746]: Uploading piece... Oct 7 09:03:11.958536798 (331h31m57.420629281s) [upload=291746]: Next returned nil segment Oct 7 09:03:11.958576012 (331h31m57.420668505s) [upload=291746]: Waiting for error group managing segments... Oct 7 09:03:12.257686376 (331h31m57.71977887s) [noise=true , address=45.91.116.4:4937 , piece_id=6KC5OU3GDXYX27JMU5XG4TRSE6MF3ITQ4FJGUZIRXOEHRYNVDPUQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:12.280245695 (331h31m57.742338178s) [noise=true , address=77.90.31.14:4662 , piece_id=GBVAUJTRKS24TNGLQ6Q7UMS6ZHP354AO7R3QAC6C5G46JLO24RRA, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:12.300371489 (331h31m57.762463972s) [noise=true , address=45.91.116.13:4153 , piece_id=VDD3KULKWJOZU2MHFA5PQISI2JLIRYEKI3OTL5VGUXL3GTO7NNMA, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:12.340028404 (331h31m57.802120888s) [noise=true , address=77.90.31.13:4626 , piece_id=ASQNXHKONSPUF75LMTT7FGQKVAUR7XGEBSITVMISEKEWFCRLJF6Q, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:12.375842663 (331h31m57.837935136s) [noise=true , address=77.90.31.12:4589 , piece_id=OL63W5YTVYOWO4HT7HDYYQIQKVYOXNAWTJPR4XIDDFO7325PHN7A, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:12.410744094 (331h31m57.872836577s) [noise=true , address=77.90.31.18:4807 , piece_id=DBD3RWNTQRDBLRUJZCFOKTFMYDCZGX4CCOHJIBWYTHFAUWIMY54A, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:12.451301444 (331h31m57.913393927s) [noise=true , address=77.90.31.21:4878 , piece_id=BA4Z5DSVFKXTIONVXEXYNCJNBKQVBZCREX5HEE4JUO2KGDI6APPQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: <nil> Oct 7 09:03:39.160144471 (331h32m24.622236965s) [noise=true , address=145.79.227.18:4006 , piece_id=SUO6MBOKAC3IBP237Z4NBXPFDUI37NVE6GRZDVBKNL5I5SGMWCCQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:32866->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:32866->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:32866->145.79.227.18:4006: use of closed network connection Oct 7 09:03:39.160252948 (331h32m24.622345421s) [noise=true , address=145.79.227.12:4767 , piece_id=ITXO2MQ5PASVTP2XDDCRFDKCRORVQY7M3JZCUPNADVS7A27X7KJQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: write tcp 188.166.225.170:54510->145.79.227.12:4767: use of closed network connection; write tcp 188.166.225.170:54510->145.79.227.12:4767: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:54510->145.79.227.12:4767: use of closed network connection Oct 7 09:03:39.160391842 (331h32m24.622484315s) [noise=true , address=145.79.227.19:4079 , piece_id=2GQDXR54ZP4YWPLMW3FARMLWDGIRDGBG7O5LKRHHZ5H2T4VCCXEQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:60370->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:60370->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60370->145.79.227.19:4079: use of closed network connection Oct 7 09:03:39.160403514 (331h32m24.622495997s) [noise=true , address=145.79.227.16:4911 , piece_id=3SRKS4TUFZQTDH7UNAUYTC2AFSP2LBP2TPTX6T2CUZQTIUCYGCWQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: read tcp 188.166.225.170:39116->145.79.227.16:4911: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:39.160566395 (331h32m24.622658868s) [noise=true , address=145.79.227.3:4151 , piece_id=2A7FM5BJTLRZCEARQEQEGQC73OJ7U5N4I5H3Q5REZO46R5SSQLYQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:145.79.227.3:4151): protocol: read tcp 188.166.225.170:56554->145.79.227.3:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:39.160586893 (331h32m24.622679366s) [noise=true , address=145.79.227.16:4909 , piece_id=BNG6I57HLTODTJQLEQ47VJMEMA3TECGDTZKBER3E5HPI3MGBO6OQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:12DaDrwzfrbz1yNwt4CpyZq7hADh5KQpJZkVmev8ukpF9wcXK2x, address:145.79.227.16:4909): protocol: read tcp 188.166.225.170:54102->145.79.227.16:4909: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:39.160746147 (331h32m24.62283862s) [noise=true , address=145.79.227.15:4876 , piece_id=A4ELUCGXMVUK6UFDTNPKKB4QC3QVJHDQ3X6CNLPVOTGP5RAJLU7Q, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:1jMZpvqeRXcQPi1XvVQpVpzU7ZkiemTZCE1yvfXDTjG4NEsmpP, address:145.79.227.15:4876): protocol: read tcp 188.166.225.170:58916->145.79.227.15:4876: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:39.160875784 (331h32m24.622968267s) [noise=true , address=145.79.227.5:4221 , piece_id=QD2QUXRE5JTWU3MCKMKBZ5C2O64BCSIRIVMHDTY7Z3I3G37SY2TQ, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: read tcp 188.166.225.170:33762->145.79.227.5:4221: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:39.160885031 (331h32m24.622977514s) [noise=true , address=145.79.227.15:4874 , piece_id=OYALXEG4UMIPI2R3HSPMVNSHSVEKXOJWDCRWX7CBHG5WRYUV6HKA, seg_pos={0 0}, upload=291746]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: read tcp 188.166.225.170:44028->145.79.227.15:4874: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:39.160915459 (331h32m24.623007942s) [seg_pos={0 0}, upload=291746]: Done waiting for segment. successful: 7 optimal: 13 errs: <nil> Oct 7 09:03:39.412580437 (331h32m24.87467291s) [upload=291746]: Done uploading segments... Oct 7 09:03:39.412609633 (331h32m24.874702106s) [upload=291746]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 7) Duration: 30.254798229s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.25596562s FAIL (2025-10-07 09:03:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== WWB6D4APLMN37MNWEFA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.143278180 (331h31m22.605370664s) [noise=true , address=145.79.227.52:4464 , piece_id=5ETOOQQOI7727FXTCCRJHD4UFC6E62FVE3O7W5E33YU6Y2QQZJQQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.143730802 (331h31m22.605823275s) [noise=true , address=145.79.227.42:4123 , piece_id=ETCFHWVTKDC6FSASDPIOHUYSULNDJDM7MTPDNDH2CJRNAMQIMRYA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.144078965 (331h31m22.606171448s) [noise=true , address=145.79.227.31:4828 , piece_id=5XUSEWAAERTABOAEPEJUOCFREK7EAZMJA5RJYFXLUK2WSO2LDRGQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.144434712 (331h31m22.606527196s) [noise=true , address=77.90.31.15:4728 , piece_id=HY7N572V2PX3NYXI6XC3CB5U6UK6IB2WASE45JWSP5VWE6ZCXHFA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.144906381 (331h31m22.606998864s) [noise=true , address=145.79.227.24:4604 , piece_id=N7H4PQI3G6EGE22KZAHARYPKQM5MVA7PMOZGLZJOC2DX5CQL72QA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.145240837 (331h31m22.60733331s) [noise=true , address=64.49.13.139:4470 , piece_id=MPHOQUIBNFPTP2N77SS5E6OQOUGKFFYSWOV57RMPXTFFC4IB5BHA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.145519819 (331h31m22.607612292s) [noise=true , address=77.90.31.21:4909 , piece_id=S3KUD7W27AJTO5LAL5K3HLICREWJJQGCTGMEL6KMIRTJOF3KXAQQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.145785565 (331h31m22.607878048s) [noise=true , address=77.90.31.12:4593 , piece_id=HLUTRAZNDV2YXRQFMWGJ2K62JVOND6L4J4JSYKCIXBEKC6RHFUXA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.146132696 (331h31m22.608225169s) [noise=true , address=145.79.227.33:4891 , piece_id=MCLZHFO2LBCIOF7OOVXHABDSG6IFLWD2URQEDPRINJNVYM4THWJA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.146145139 (331h31m22.608237622s) [noise=true , address=77.90.31.13:4642 , piece_id=CQKDF5DMZF4E4B7DW6MED2QOXU25GVB77L4LMOZFBWZ3A6X5D7FQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.146484165 (331h31m22.608576648s) [noise=true , address=45.91.116.10:4106 , piece_id=3JEX2R2O5JGUCFYZ3SYRDD54UDXNDR45RCQC55ZKFA2N2PEYTPSA, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.146854481 (331h31m22.608946954s) [noise=true , address=77.90.31.11:4556 , piece_id=HXKOXK7TPW6WLZY5BXXMCCG2UP4QM6TEPWIFBR3QIRALFGTYEJNQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.147166986 (331h31m22.609259469s) [noise=true , address=77.90.31.10:4540 , piece_id=SCMMESTRCKCIARYSSSNZRZ2SLQKZG7PJKWZMJCBMLHKYUVGYAGXQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.147480322 (331h31m22.609572805s) [noise=true , address=45.91.116.9:4274 , piece_id=KP5EXNWXQOI2GJEEUWBIS5OMBJQB5JIRA7WJCCBZD7DCQKN4NL2A, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:37.147834587 (331h31m22.60992707s) [noise=true , address=45.91.116.5:4318 , piece_id=KN7IWMODJEYCBJ3WGOLKDWFOCINZCXOP3P4QTNVRXLYXMD56JJQQ, seg_pos={0 0}, upload=291737]: Uploading piece... Oct 7 09:02:38.482809506 (331h31m23.944901989s) [upload=291737]: Next returned nil segment Oct 7 09:02:38.482828262 (331h31m23.944920735s) [upload=291737]: Waiting for error group managing segments... Oct 7 09:02:38.825769931 (331h31m24.287862415s) [noise=true , address=64.49.13.139:4470 , piece_id=MPHOQUIBNFPTP2N77SS5E6OQOUGKFFYSWOV57RMPXTFFC4IB5BHA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.118235693 (331h31m24.580328176s) [noise=true , address=45.91.116.9:4274 , piece_id=KP5EXNWXQOI2GJEEUWBIS5OMBJQB5JIRA7WJCCBZD7DCQKN4NL2A, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.140867440 (331h31m24.602959923s) [noise=true , address=45.91.116.10:4106 , piece_id=3JEX2R2O5JGUCFYZ3SYRDD54UDXNDR45RCQC55ZKFA2N2PEYTPSA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.181414300 (331h31m24.643506773s) [noise=true , address=77.90.31.21:4909 , piece_id=S3KUD7W27AJTO5LAL5K3HLICREWJJQGCTGMEL6KMIRTJOF3KXAQQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.195709212 (331h31m24.657801695s) [noise=true , address=45.91.116.5:4318 , piece_id=KN7IWMODJEYCBJ3WGOLKDWFOCINZCXOP3P4QTNVRXLYXMD56JJQQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.195887912 (331h31m24.657980395s) [noise=true , address=77.90.31.10:4540 , piece_id=SCMMESTRCKCIARYSSSNZRZ2SLQKZG7PJKWZMJCBMLHKYUVGYAGXQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.209543024 (331h31m24.671635497s) [noise=true , address=77.90.31.13:4642 , piece_id=CQKDF5DMZF4E4B7DW6MED2QOXU25GVB77L4LMOZFBWZ3A6X5D7FQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.225725478 (331h31m24.687817961s) [noise=true , address=77.90.31.11:4556 , piece_id=HXKOXK7TPW6WLZY5BXXMCCG2UP4QM6TEPWIFBR3QIRALFGTYEJNQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.232975715 (331h31m24.695068188s) [noise=true , address=77.90.31.15:4728 , piece_id=HY7N572V2PX3NYXI6XC3CB5U6UK6IB2WASE45JWSP5VWE6ZCXHFA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:02:39.255527830 (331h31m24.717620333s) [noise=true , address=77.90.31.12:4593 , piece_id=HLUTRAZNDV2YXRQFMWGJ2K62JVOND6L4J4JSYKCIXBEKC6RHFUXA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: <nil> Oct 7 09:03:36.185943018 (331h32m21.648035491s) [noise=true , address=145.79.227.31:4828 , piece_id=5XUSEWAAERTABOAEPEJUOCFREK7EAZMJA5RJYFXLUK2WSO2LDRGQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:1cX7FtKC8KMopnX9zvkT6fYdnutK84cFjLW6xMwpXuGsqpiN8k, address:145.79.227.31:4828): protocol: write tcp 188.166.225.170:59578->145.79.227.31:4828: use of closed network connection; write tcp 188.166.225.170:59578->145.79.227.31:4828: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59578->145.79.227.31:4828: use of closed network connection Oct 7 09:03:36.186092803 (331h32m21.648185286s) [noise=true , address=145.79.227.34:4917 , piece_id=3W7XTDQQFEKXGEXCONK2UEB3JRORIRUCYWWB6D4APLMN37MNWEFA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:1aiK1biSRwVPTPj2CBtYggcgpEGQthN8zx5YWcz4viSwDs1TXZ, address:145.79.227.34:4917): protocol: read tcp 188.166.225.170:32818->145.79.227.34:4917: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186169550 (331h32m21.648262053s) [noise=true , address=145.79.227.24:4604 , piece_id=N7H4PQI3G6EGE22KZAHARYPKQM5MVA7PMOZGLZJOC2DX5CQL72QA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:12EhoYFXjEa1ciTS2UXFRFUWhfSyd7p3oPDgqrdNVTYyBUk4mJ1, address:145.79.227.24:4604): protocol: read tcp 188.166.225.170:49698->145.79.227.24:4604: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186211650 (331h32m21.648304133s) [noise=true , address=145.79.227.42:4123 , piece_id=ETCFHWVTKDC6FSASDPIOHUYSULNDJDM7MTPDNDH2CJRNAMQIMRYA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:12ttjngUh34tGJpCBgVCLfqrWwsdmd88XKWPrmaJrgkMu8htkKM, address:145.79.227.42:4123): protocol: read tcp 188.166.225.170:60884->145.79.227.42:4123: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186302652 (331h32m21.648395125s) [noise=true , address=145.79.227.22:4519 , piece_id=HBDWJQNVH65VBA3HLUYFMQ2Z56HSWIPNZGNRQNKS2OLSLU66Z7JA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:1p5DYJfayCq68BQGnSEAoZCri8TWpST5WNcx5QFvzcxLyFamnj, address:145.79.227.22:4519): protocol: read tcp 188.166.225.170:52076->145.79.227.22:4519: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186418644 (331h32m21.648511117s) [noise=true , address=145.79.227.23:4559 , piece_id=X566EYFISZU3XE36PM5IW2RVL32VXX7OC6BAP7BCFSS4GRLMIKRA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:1XXGQfMeXYCUuSh5TUe8pTwwd19wn68r9buS11N8aJdx6vZysf, address:145.79.227.23:4559): protocol: read tcp 188.166.225.170:39522->145.79.227.23:4559: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186498155 (331h32m21.648590638s) [noise=true , address=145.79.227.52:4464 , piece_id=5ETOOQQOI7727FXTCCRJHD4UFC6E62FVE3O7W5E33YU6Y2QQZJQQ, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:1mLbGHntspYmsTVg8433HYSdAEMrXjGnDhmmvX1JmmgcxtdLgk, address:145.79.227.52:4464): protocol: read tcp 188.166.225.170:48390->145.79.227.52:4464: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186527411 (331h32m21.648619894s) [noise=true , address=145.79.227.33:4891 , piece_id=MCLZHFO2LBCIOF7OOVXHABDSG6IFLWD2URQEDPRINJNVYM4THWJA, seg_pos={0 0}, upload=291737]: Done uploading piece. err: ecclient: upload failed (node:1RHLM4MJmtKoHsUnsqccy1ehyD7zVSn6gnJNp7yJJmZhTJqPkU, address:145.79.227.33:4891): protocol: read tcp 188.166.225.170:50354->145.79.227.33:4891: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:03:36.186556196 (331h32m21.648648669s) [seg_pos={0 0}, upload=291737]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Oct 7 09:03:36.439967229 (331h32m21.902059712s) [upload=291737]: Done uploading segments... Oct 7 09:03:36.440022082 (331h32m21.902114566s) [upload=291737]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 1m0.255994073s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [...................................................X......XX] duration: 30.245840277s FAIL (2025-10-07 09:02:56 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Oct 7 09:02:27.402616705 (331h31m12.864709188s) [upload=291735]: Starting upload Oct 7 09:02:27.402708790 (331h31m12.864801283s) [upload=291735]: Done starting upload Oct 7 09:02:27.402840571 (331h31m12.864933054s) [upload=291735]: Uploading segments... Oct 7 09:02:27.403989098 (331h31m12.866081581s) [upload=291735]: Got next segment. Inline: false Oct 7 09:02:27.404033011 (331h31m12.866125494s) [upload=291735]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:02:27.404038271 (331h31m12.866130754s) [upload=291735]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:02:27.655548725 (331h31m13.117641209s) [seg_pos={0 0}, upload=291735]: Begin upload segment... Oct 7 09:02:27.655626103 (331h31m13.117718586s) [seg_pos={0 0}, upload=291735]: Done begin upload segment. Oct 7 09:02:27.655678081 (331h31m13.117770554s) [noise=true , address=77.90.31.21:4878 , piece_id=JRM2CWVFAJYHWVMKG5I76KKJ62RSKGLIWOPMK2XYERUNIXY3WSGQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.655749136 (331h31m13.117841619s) [noise=true , address=77.90.31.11:4553 , piece_id=5J3CHSDBL3QBVBPUTEKPNW2OGKD7IYCKC2MAUH4DHXWMJKP7FD7A, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.656446916 (331h31m13.118539389s) [noise=true , address=145.79.227.18:4007 , piece_id=34LAU2BYRAJCJJKPNBQN4DHK6WFK7IOCSEAM4TYIJ5WJSUS5V2IQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.657628306 (331h31m13.119720789s) [noise=true , address=45.91.116.7:4011 , piece_id=22GJDJTUH7OVPWLYLLF2ZXCPA4DVJLEDKFDBFX7C6IRQJ7OPEGNA, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.658458176 (331h31m13.120550659s) [noise=true , address=77.90.31.10:4517 , piece_id=B2FFYFNTFQGVFILFNRAUYS5EANINGJIUXFD6FYNHHY4DSPEBLIWQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.659352809 (331h31m13.121445292s) [noise=true , address=45.91.116.13:4153 , piece_id=JAIFLQGQPUEGLX7RNPTMDT6LKQCTFVUZEHWJU5HG3DOYUX4LOICQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.660469836 (331h31m13.122562309s) [noise=true , address=77.90.31.14:4661 , piece_id=CGNIPNMK72IUG5AMR3TSZGKIDDUEOYMZPA2QHIDFCFLIK2S23EVQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.662054764 (331h31m13.124147247s) [noise=true , address=145.79.227.11:4730 , piece_id=D66HIOIWKSUAGSV4MFIDVZS3M7GGWJLIQB4CAIWH6V5OJ3DB4WQA, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.666108252 (331h31m13.128200735s) [noise=true , address=145.79.227.2:4113 , piece_id=2F66RKQLEDIUNERP3O3SJEV5TRVAGGCHSADWBSO5VTTZUXPR6R6A, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.666449061 (331h31m13.128541545s) [noise=true , address=145.79.227.13:4802 , piece_id=LDOL5MCGM56HNFLSWEHMXCVF7MHWXFAFWU4VK56J2HNGCDC3LDYQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.672452954 (331h31m13.134545438s) [noise=true , address=45.91.116.14:4191 , piece_id=MRD2XNTCCOTDM2C3VW5LCHRC4COH6Y65MKJDQ3CI6TNWNXKRFMIQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.675171421 (331h31m13.137263904s) [noise=true , address=45.91.116.8:4047 , piece_id=Y46T36HLNK4DWM4OD2S6QBD5EBQQPSVZYP77GJVXCZMX5WB5CM7A, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.675724634 (331h31m13.137817127s) [noise=true , address=45.91.116.14:4190 , piece_id=JSHZYN75R5DAPSKRX2NUONTIVURF5M54MEN7JHPWBUXW4ACYHIEQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.678944396 (331h31m13.141036879s) [noise=true , address=145.79.227.3:4150 , piece_id=4BRUOI7U6GCWSJF6QXRVRL3532NGVL42C4KCO5GUAUQMKYCKU2EQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.681140207 (331h31m13.14323269s) [noise=true , address=45.91.116.4:4940 , piece_id=QCOQRJCJIGKOF73ICRLUVMJNJGPTXC4CSLRFKE666LRMQUA4GL2A, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:27.688176346 (331h31m13.150268839s) [noise=true , address=45.91.116.4:4937 , piece_id=WG2GRTLREM7AOVE5D56MOTJXPOJ3UUZBNIWVEIVPUOYZWXN233KQ, seg_pos={0 0}, upload=291735]: Uploading piece... Oct 7 09:02:29.418500951 (331h31m14.880593434s) [upload=291735]: Next returned nil segment Oct 7 09:02:29.418521320 (331h31m14.880613803s) [upload=291735]: Waiting for error group managing segments... Oct 7 09:02:29.768623705 (331h31m15.230716188s) [noise=true , address=45.91.116.14:4190 , piece_id=JSHZYN75R5DAPSKRX2NUONTIVURF5M54MEN7JHPWBUXW4ACYHIEQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.782233832 (331h31m15.244326305s) [noise=true , address=77.90.31.14:4661 , piece_id=CGNIPNMK72IUG5AMR3TSZGKIDDUEOYMZPA2QHIDFCFLIK2S23EVQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.796356656 (331h31m15.258449139s) [noise=true , address=45.91.116.4:4940 , piece_id=QCOQRJCJIGKOF73ICRLUVMJNJGPTXC4CSLRFKE666LRMQUA4GL2A, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.801067926 (331h31m15.263160409s) [noise=true , address=45.91.116.7:4011 , piece_id=22GJDJTUH7OVPWLYLLF2ZXCPA4DVJLEDKFDBFX7C6IRQJ7OPEGNA, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.813010518 (331h31m15.275103001s) [noise=true , address=45.91.116.4:4937 , piece_id=WG2GRTLREM7AOVE5D56MOTJXPOJ3UUZBNIWVEIVPUOYZWXN233KQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.828466720 (331h31m15.290559203s) [noise=true , address=77.90.31.11:4553 , piece_id=5J3CHSDBL3QBVBPUTEKPNW2OGKD7IYCKC2MAUH4DHXWMJKP7FD7A, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.835137844 (331h31m15.297230317s) [noise=true , address=45.91.116.8:4047 , piece_id=Y46T36HLNK4DWM4OD2S6QBD5EBQQPSVZYP77GJVXCZMX5WB5CM7A, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.837549546 (331h31m15.299642029s) [noise=true , address=77.90.31.10:4517 , piece_id=B2FFYFNTFQGVFILFNRAUYS5EANINGJIUXFD6FYNHHY4DSPEBLIWQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.846604609 (331h31m15.308697142s) [noise=true , address=77.90.31.21:4878 , piece_id=JRM2CWVFAJYHWVMKG5I76KKJ62RSKGLIWOPMK2XYERUNIXY3WSGQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.865534616 (331h31m15.327627099s) [noise=true , address=45.91.116.13:4153 , piece_id=JAIFLQGQPUEGLX7RNPTMDT6LKQCTFVUZEHWJU5HG3DOYUX4LOICQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:29.913018246 (331h31m15.375110729s) [noise=true , address=45.91.116.14:4191 , piece_id=MRD2XNTCCOTDM2C3VW5LCHRC4COH6Y65MKJDQ3CI6TNWNXKRFMIQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: <nil> Oct 7 09:02:56.702433045 (331h31m42.164525518s) [noise=true , address=145.79.227.2:4113 , piece_id=2F66RKQLEDIUNERP3O3SJEV5TRVAGGCHSADWBSO5VTTZUXPR6R6A, seg_pos={0 0}, upload=291735]: Done uploading piece. err: ecclient: upload failed (node:1SQKETWozrz4PcDC3yhy6V5d8Ryo6NNiyWWEkE5xRcmUj9B6UV, address:145.79.227.2:4113): protocol: read tcp 188.166.225.170:48360->145.79.227.2:4113: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:56.702560297 (331h31m42.16465277s) [noise=true , address=145.79.227.13:4802 , piece_id=LDOL5MCGM56HNFLSWEHMXCVF7MHWXFAFWU4VK56J2HNGCDC3LDYQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: ecclient: upload failed (node:1v2RSYwuQZPTLdWGz4QEeEa9STJy1X6HkexD4RYkVjTHGrNcx4, address:145.79.227.13:4802): protocol: read tcp 188.166.225.170:37848->145.79.227.13:4802: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:56.702730141 (331h31m42.164822614s) [noise=true , address=145.79.227.18:4007 , piece_id=34LAU2BYRAJCJJKPNBQN4DHK6WFK7IOCSEAM4TYIJ5WJSUS5V2IQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: ecclient: upload failed (node:19A6vk5g5ajUnXEtmXKDb9TqtjEBLYEdDvwqYsCtXKdhYUxrnp, address:145.79.227.18:4007): protocol: read tcp 188.166.225.170:54664->145.79.227.18:4007: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:56.702739719 (331h31m42.164832192s) [noise=true , address=145.79.227.11:4730 , piece_id=D66HIOIWKSUAGSV4MFIDVZS3M7GGWJLIQB4CAIWH6V5OJ3DB4WQA, seg_pos={0 0}, upload=291735]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: read tcp 188.166.225.170:47400->145.79.227.11:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:56.702834970 (331h31m42.164927453s) [noise=true , address=145.79.227.3:4150 , piece_id=4BRUOI7U6GCWSJF6QXRVRL3532NGVL42C4KCO5GUAUQMKYCKU2EQ, seg_pos={0 0}, upload=291735]: Done uploading piece. err: ecclient: upload failed (node:12YanYT7FkCt7SzDuqx15RP9zxaYmikVLzPkBp4i15WEShDpfpK, address:145.79.227.3:4150): protocol: read tcp 188.166.225.170:42916->145.79.227.3:4150: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:56.702876229 (331h31m42.164968702s) [seg_pos={0 0}, upload=291735]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Oct 7 09:02:56.945668281 (331h31m42.407760784s) [upload=291735]: Done uploading segments... Oct 7 09:02:56.945707696 (331h31m42.407800179s) [upload=291735]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.245863982s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X......X] duration: 30.249226932s FAIL (2025-10-07 09:02:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 9) ======LOGS====== oad Oct 7 09:01:52.131723436 (331h30m37.593815919s) [upload=291728]: Done starting upload Oct 7 09:01:52.132006967 (331h30m37.59409945s) [upload=291728]: Uploading segments... Oct 7 09:01:52.133287866 (331h30m37.595380349s) [upload=291728]: Got next segment. Inline: false Oct 7 09:01:52.133352288 (331h30m37.595444761s) [upload=291728]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 09:01:52.133359252 (331h30m37.595451735s) [upload=291728]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 09:01:52.387317827 (331h30m37.84941031s) [seg_pos={0 0}, upload=291728]: Begin upload segment... Oct 7 09:01:52.387404312 (331h30m37.849496795s) [seg_pos={0 0}, upload=291728]: Done begin upload segment. Oct 7 09:01:52.387444559 (331h30m37.849537042s) [noise=true , address=145.79.227.19:4077 , piece_id=PGTIX75PR66G25ZO4ZYII3OKHCCTYTLDC2DSREKHOGUGGN67AOWQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.387469526 (331h30m37.849562009s) [noise=true , address=77.90.31.15:4698 , piece_id=7BEYZA2OISQMRMLOCKY24LO7OOCMPNPKODEWZZQNOQ4LTNTVROEQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.387630924 (331h30m37.849723407s) [noise=true , address=77.90.31.12:4589 , piece_id=64YIAI2F7FO4Y7HWMMZQB76V7NCFNH6S5M4FEPW2ZD3MJ5D7HTYQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.387708441 (331h30m37.849800924s) [noise=true , address=145.79.227.13:4804 , piece_id=DIGB5WZUW5VUAAAFA62AFRABIQVQXGBSST43OLHHR3PUO3WW53NQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.388356636 (331h30m37.850449109s) [noise=true , address=45.91.116.9:4263 , piece_id=HUF7XHP5P33ML3GABPLQGRU4L2ZXQKTU64VNFAXUVYR6NIDZYDOQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.388727231 (331h30m37.850819704s) [noise=true , address=45.91.116.14:4189 , piece_id=EZNPHWZEKSIGS5Z7MYGI6T3UDTKJYZW27XOV3YSHPDXPMJS5G6KQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.389210361 (331h30m37.851302844s) [noise=true , address=145.79.227.11:4730 , piece_id=33CRK6DZNVVGZ3WBTTGJZ3WLHDAOU4AL56TJXI4PDUF6G54LKQIQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.389221653 (331h30m37.851314126s) [noise=true , address=145.79.227.16:4912 , piece_id=JH6RWIHZTC6NGGUCHRGI62DEV2RCF6JIWOWRUW32TCU3VYNLKDSQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.389743567 (331h30m37.85183604s) [noise=true , address=145.79.227.12:4767 , piece_id=H2XWHA4Y673IEWBUTZHG35OP5RSF3PK777QIG4EGBLSI6GHENEZQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.390224983 (331h30m37.852317466s) [noise=true , address=77.90.31.16:4734 , piece_id=BDDMZW3GX6DJVLQFDAA3MAVMKRAZSMFE67D2EA4UB3BFQN6ZL55Q, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.390330364 (331h30m37.852422847s) [noise=true , address=145.79.227.14:4838 , piece_id=XALOAQOSKY34ZLI5V6E5LU2IPOUHLB7YUNV2PWNUQRPWLBBH3LHA, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.390903956 (331h30m37.852996439s) [noise=true , address=145.79.227.4:4186 , piece_id=EKOYLA7JMMY2UJFAHEAJH6WZJNS7PMMSAJ3QXDE2MACH7X2YJF2Q, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.391367388 (331h30m37.853459871s) [noise=true , address=77.90.31.21:4878 , piece_id=HYHD4ZYL5SYTDCVP6G5XY7AX6THD7CVHVZL6K7IV2LMTQT2EQ2KA, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.391838145 (331h30m37.853930618s) [noise=true , address=77.90.31.19:4842 , piece_id=HY2HYP7CAY5VGOTBTW6QJNUEXKDDIQ5CCAYY42KGSBCGXDHBRTGQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.392343458 (331h30m37.854435931s) [noise=true , address=45.91.116.9:4262 , piece_id=UQBC3YWPT5BUKUVOUC4ETA6XOTBWJHC4J2SUSEVE6JLD4X2JE35A, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:52.392948469 (331h30m37.855040952s) [noise=true , address=77.90.31.21:4879 , piece_id=ODLZRYLPYU6S24KEXNMZ6X3B3SU7MX3RK4GRSGEAU2QQSTPBIIPQ, seg_pos={0 0}, upload=291728]: Uploading piece... Oct 7 09:01:54.107909450 (331h30m39.570001923s) [upload=291728]: Next returned nil segment Oct 7 09:01:54.107928355 (331h30m39.570020838s) [upload=291728]: Waiting for error group managing segments... Oct 7 09:01:54.436961570 (331h30m39.899054053s) [noise=true , address=77.90.31.12:4589 , piece_id=64YIAI2F7FO4Y7HWMMZQB76V7NCFNH6S5M4FEPW2ZD3MJ5D7HTYQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.449961575 (331h30m39.912054058s) [noise=true , address=77.90.31.19:4842 , piece_id=HY2HYP7CAY5VGOTBTW6QJNUEXKDDIQ5CCAYY42KGSBCGXDHBRTGQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.457295731 (331h30m39.919388214s) [noise=true , address=77.90.31.21:4878 , piece_id=HYHD4ZYL5SYTDCVP6G5XY7AX6THD7CVHVZL6K7IV2LMTQT2EQ2KA, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.473134030 (331h30m39.935226514s) [noise=true , address=45.91.116.9:4262 , piece_id=UQBC3YWPT5BUKUVOUC4ETA6XOTBWJHC4J2SUSEVE6JLD4X2JE35A, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.499589428 (331h30m39.961681901s) [noise=true , address=45.91.116.14:4189 , piece_id=EZNPHWZEKSIGS5Z7MYGI6T3UDTKJYZW27XOV3YSHPDXPMJS5G6KQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.509358470 (331h30m39.971450954s) [noise=true , address=45.91.116.9:4263 , piece_id=HUF7XHP5P33ML3GABPLQGRU4L2ZXQKTU64VNFAXUVYR6NIDZYDOQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.525972928 (331h30m39.988065411s) [noise=true , address=77.90.31.21:4879 , piece_id=ODLZRYLPYU6S24KEXNMZ6X3B3SU7MX3RK4GRSGEAU2QQSTPBIIPQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.594638524 (331h30m40.056730997s) [noise=true , address=77.90.31.15:4698 , piece_id=7BEYZA2OISQMRMLOCKY24LO7OOCMPNPKODEWZZQNOQ4LTNTVROEQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:01:54.606080672 (331h30m40.068173145s) [noise=true , address=77.90.31.16:4734 , piece_id=BDDMZW3GX6DJVLQFDAA3MAVMKRAZSMFE67D2EA4UB3BFQN6ZL55Q, seg_pos={0 0}, upload=291728]: Done uploading piece. err: <nil> Oct 7 09:02:21.418389421 (331h31m6.880481905s) [noise=true , address=145.79.227.12:4767 , piece_id=H2XWHA4Y673IEWBUTZHG35OP5RSF3PK777QIG4EGBLSI6GHENEZQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:1TiDHs6xp3atLDHHv1i2564ibyPGLKfFyHx3bgZWGRRN5cwUi5, address:145.79.227.12:4767): protocol: read tcp 188.166.225.170:36190->145.79.227.12:4767: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418516133 (331h31m6.880608606s) [noise=true , address=145.79.227.14:4838 , piece_id=XALOAQOSKY34ZLI5V6E5LU2IPOUHLB7YUNV2PWNUQRPWLBBH3LHA, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:145.79.227.14:4838): protocol: read tcp 188.166.225.170:51592->145.79.227.14:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418611555 (331h31m6.880704038s) [noise=true , address=145.79.227.4:4186 , piece_id=EKOYLA7JMMY2UJFAHEAJH6WZJNS7PMMSAJ3QXDE2MACH7X2YJF2Q, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: read tcp 188.166.225.170:36946->145.79.227.4:4186: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418672131 (331h31m6.880764614s) [noise=true , address=145.79.227.11:4730 , piece_id=33CRK6DZNVVGZ3WBTTGJZ3WLHDAOU4AL56TJXI4PDUF6G54LKQIQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:12T7G85LguKGbxuMDp5u7q1j3TWc1ERBH6YgynJhE5NzzcyuMom, address:145.79.227.11:4730): protocol: read tcp 188.166.225.170:54022->145.79.227.11:4730: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418794213 (331h31m6.880886686s) [noise=true , address=145.79.227.16:4912 , piece_id=JH6RWIHZTC6NGGUCHRGI62DEV2RCF6JIWOWRUW32TCU3VYNLKDSQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:1gwuDRBC5oJXgZbP6FqP9tc8UB6aJt2uexUgSuzHCovw27v1NR, address:145.79.227.16:4912): protocol: read tcp 188.166.225.170:48962->145.79.227.16:4912: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418892119 (331h31m6.880984592s) [noise=true , address=145.79.227.13:4804 , piece_id=DIGB5WZUW5VUAAAFA62AFRABIQVQXGBSST43OLHHR3PUO3WW53NQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:145.79.227.13:4804): protocol: read tcp 188.166.225.170:54186->145.79.227.13:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418925954 (331h31m6.881018437s) [noise=true , address=145.79.227.19:4077 , piece_id=PGTIX75PR66G25ZO4ZYII3OKHCCTYTLDC2DSREKHOGUGGN67AOWQ, seg_pos={0 0}, upload=291728]: Done uploading piece. err: ecclient: upload failed (node:12kDJuQP8iLUjB1rXpcNX5TtHzsqa5HvYcUD9Ef64HYTnTXcwLk, address:145.79.227.19:4077): protocol: read tcp 188.166.225.170:59514->145.79.227.19:4077: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Oct 7 09:02:21.418953376 (331h31m6.881045849s) [seg_pos={0 0}, upload=291728]: Done waiting for segment. successful: 9 optimal: 13 errs: <nil> Oct 7 09:02:21.665316838 (331h31m7.127409322s) [upload=291728]: Done uploading segments... Oct 7 09:02:21.665348609 (331h31m7.127441092s) [upload=291728]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 9) Duration: 30.249257268s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.008133684s FAIL (2025-10-07 09:01:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.008164993s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.192244945s FAIL (2025-10-07 09:00:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.192271026s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.123226404s FAIL (2025-10-07 08:59:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.123260248s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.153245913s FAIL (2025-10-07 08:59:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.15344451s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.085950496s FAIL (2025-10-07 08:58:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.085977047s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.409934044s FAIL (2025-10-07 08:58:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.409954013s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 52.989137284s FAIL (2025-10-07 08:58:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Oct 7 08:57:29.345050413 (331h26m14.807142896s) [upload=291663]: Starting upload Oct 7 08:57:29.345146566 (331h26m14.807239049s) [upload=291663]: Done starting upload Oct 7 08:57:29.345372406 (331h26m14.807464889s) [upload=291663]: Uploading segments... Oct 7 08:57:29.345529886 (331h26m14.807622369s) [upload=291663]: Got next segment. Inline: false Oct 7 08:57:29.345586944 (331h26m14.807679427s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectBegin Oct 7 08:57:29.345595862 (331h26m14.807688335s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentBegin Oct 7 08:57:29.606203899 (331h26m15.068296382s) [seg_pos={0 0}, upload=291663]: Begin upload segment... Oct 7 08:57:29.606273582 (331h26m15.068366055s) [seg_pos={0 0}, upload=291663]: Done begin upload segment. Oct 7 08:57:29.606307286 (331h26m15.068399759s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606319940 (331h26m15.068412413s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606365688 (331h26m15.068458171s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606434499 (331h26m15.068526982s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.606893843 (331h26m15.068986326s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607273566 (331h26m15.069366049s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.607695079 (331h26m15.069787562s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608173219 (331h26m15.070265693s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608184671 (331h26m15.070277154s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608578711 (331h26m15.070671184s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.608829839 (331h26m15.070922322s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609115704 (331h26m15.071208187s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609540463 (331h26m15.071632946s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.609970952 (331h26m15.072063435s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.610316781 (331h26m15.072409254s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:29.611728139 (331h26m15.073820622s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Uploading piece... Oct 7 08:57:31.316440855 (331h26m16.778533328s) [upload=291663]: Next returned nil segment Oct 7 08:57:31.316458368 (331h26m16.778550851s) [upload=291663]: Waiting for error group managing segments... Oct 7 08:57:31.594867042 (331h26m17.056959525s) [noise=true , address=145.79.227.16:4909 , piece_id=3CDR7A5VYQTKWPWGXUEC2ZEBPZZMTYEZ7LPYHLPJF3VIOISWMVYQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.594931505 (331h26m17.057023988s) [noise=true , address=145.79.227.16:4912 , piece_id=JCBEKZAA3EW62FDZ3YINZQHMQJ6DFTQAG3T4Q4VPMBJJFYH4FQBQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.606515934 (331h26m17.068608417s) [noise=true , address=145.79.227.14:4839 , piece_id=OV6HDEBIH7X2R5GU6CZRMDVVJSJR6SIL4R3F6TS3UYLZNEUGCCGA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.610473129 (331h26m17.072565612s) [noise=true , address=145.79.227.17:4969 , piece_id=4TZEHWOE6BGWP36NATFL2H7ZJULR3EVZ7GMNOZRCPPNXQT2764BQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.702719277 (331h26m17.16481175s) [noise=true , address=45.91.116.8:4047 , piece_id=ZP2SEGFKKGXMOVCUWW7FKC7VKJ5A7IVQWCL2TMLFOYBXGSZNS5VQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.706384838 (331h26m17.168477321s) [noise=true , address=45.91.116.4:4937 , piece_id=JDHXWOX4ZXX556LOGGAHCN56TBSN4QVYYVPVP3MDSJR4BFPK2JFA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.741976331 (331h26m17.204068815s) [noise=true , address=45.91.116.7:4010 , piece_id=PL2AIXDZR4XNWDSMLTBK2FV5ZWA76JTBY6GHQJ24XDMIDUAV7K6Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.753572834 (331h26m17.215665317s) [noise=true , address=77.90.31.14:4661 , piece_id=KMJ4Q7RAWJCULHEKN4BZ5OCRTSAACTICPTY36CC3D4WZKR44P76Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.765491029 (331h26m17.227583512s) [noise=true , address=45.91.116.8:4045 , piece_id=OBFTATWVJQEQEM2LN3SXK4IPHFIAJBNQOJGLWP5Z6EK6LRXMIGZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.793605535 (331h26m17.255698008s) [noise=true , address=45.91.116.6:4975 , piece_id=Z2NPKHP62RS3P53FDF2RETXDHZLB6DFXCLNG4NENWBH32B2YNQRQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.813647199 (331h26m17.275739672s) [noise=true , address=45.91.116.7:4009 , piece_id=AGTE465DOE3HXF2LNFB45HBFLX5WO7UWAHDHCK42SJ6QXLISFNVQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821160076 (331h26m17.283252569s) [noise=true , address=45.91.116.12:4226 , piece_id=VBPIKJWAGEW2JCQQVFNNQCTO57QAGP4TWZPUCUPWCZPGFLXRJDDA, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821546712 (331h26m17.283639195s) [noise=true , address=45.91.116.14:4190 , piece_id=MN7ZFNE54GR5XZYCGANZFMYKVPPSO2DL5GXXG3WSAEHVFG5FSF6A, seg_pos={0 0}, upload=291663]: Done uploading piece. err: <nil> Oct 7 08:57:31.821587660 (331h26m17.283680153s) [seg_pos={0 0}, upload=291663]: Segment reached optimal threshold of 13 pieces. Oct 7 08:57:31.821758786 (331h26m17.283851279s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1eto4dNcMUufd2GixDpEcuXN9ocPRFdnJUPht7vBRdA3BcqRek): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821780287 (331h26m17.283872771s) [noise=true , address=77.90.31.11:4554 , piece_id=2GOFZ42Z7HWPYMIZ2F4BSNK2ISIJTJZXK4KNKQWRHR3HOLYZ4YZQ, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.821840892 (331h26m17.283933365s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12AxqL8pD57cAVRsoNMBsqjWCvRwnKYwb9ESatQQZLRxFEGCjq2): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.821848546 (331h26m17.28394103s) [noise=true , address=77.90.31.17:4771 , piece_id=6KUQEHA5M33AJMBBITVIYHNWGBR6VRGG5EKMS5WCJT7MXSTTDU5Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822003382 (331h26m17.284095865s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ERDD2hvVwWnBGdG7C1aCWMZGzbGHtKeKio5fwRkmo5f8fms8a): protocol: expected piece hash; context canceled; EOF Oct 7 08:57:31.822014713 (331h26m17.284107186s) [noise=true , address=77.90.31.13:4628 , piece_id=6MOSV3FOKZK3CSPOALTEHM4HT3KBJOIEYEIJPE4PBUTF6KAHU36Q, seg_pos={0 0}, upload=291663]: Long tail cancellation due to optimal threshold Oct 7 08:57:31.822089385 (331h26m17.284181858s) [seg_pos={0 0}, upload=291663]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Oct 7 08:57:31.822301208 (331h26m17.284393702s) [upload=291663]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 08:57:31.822310337 (331h26m17.28440283s) [upload=291663]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 08:58:21.608482156 (331h27m7.070574629s) [upload=291663]: Done uploading segments... Oct 7 08:58:21.608520760 (331h27m7.070613243s) [upload=291663]: Upload finished. err: metaclient: context deadline exceeded Duration: 52.989164726s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.681616132s FAIL (2025-10-07 08:33:20 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.681657641s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.617814641s FAIL (2025-10-07 08:32:30 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.617856041s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.880978631s FAIL (2025-10-07 08:32:04 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.881016673s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.667717785s FAIL (2025-10-07 08:31:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.667757451s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.721150136s FAIL (2025-10-07 08:30:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.721186625s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.716576873s FAIL (2025-10-07 08:30:36 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.716594115s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.002515883s FAIL (2025-10-07 08:30:14 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 6.002552051s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.713306589s FAIL (2025-10-07 08:30:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.713345553s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.150931954s FAIL (2025-10-07 08:29:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.150958487s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.817594463s FAIL (2025-10-07 08:29:47 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.817626262s core uplink us1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.22826209s FAIL (2025-10-07 08:29:30 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.228290965s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.764162186s FAIL (2025-10-07 08:28:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.764190839s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.435283896s FAIL (2025-10-07 08:28:52 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.4353131s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.807353732s FAIL (2025-10-07 08:28:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 5.807427643s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.121608957s FAIL (2025-10-07 08:28:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.121639103s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 11.046810125s FAIL (2025-10-07 08:27:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 11.046841886s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.636709451s FAIL (2025-10-07 08:27:50 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.636739618s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 7.950446264s FAIL (2025-10-07 08:27:32 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 2.684228524 (330h16m48.146321007s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684385934 (330h16m48.146478417s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.684963844 (330h16m48.147056327s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685221956 (330h16m48.147314439s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685452474 (330h16m48.147544957s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685900778 (330h16m48.147993261s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.685950814 (330h16m48.148043297s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686310157 (330h16m48.148402631s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.686461365 (330h16m48.148553848s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687006704 (330h16m48.149099187s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687022113 (330h16m48.149114596s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687498480 (330h16m48.149590963s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.687559176 (330h16m48.149651699s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.688272183 (330h16m48.150364666s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690143407 (330h16m48.15223589s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690606197 (330h16m48.152698671s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:02.690968368 (330h16m48.153060851s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Uploading piece... Oct 7 07:48:04.184255931 (330h16m49.646348424s) [upload=290665]: Next returned nil segment Oct 7 07:48:04.184283844 (330h16m49.646376327s) [upload=290665]: Waiting for error group managing segments... Oct 7 07:48:04.558544184 (330h16m50.020636667s) [noise=true , address=145.79.227.22:4512 , piece_id=ZHYRXIBGXVLWMXK3JSTX5BM3KK3UYASIGFEA3HIKQEXWRPOXZKDA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.559842245 (330h16m50.021934739s) [noise=true , address=145.79.227.30:4777 , piece_id=UX4R7UC5EEX67KDL7SVKFPWD4VVB4WJPKAIBSRY5Y4IMAKRVKXYA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.567228070 (330h16m50.029320553s) [noise=true , address=145.79.227.23:4561 , piece_id=2UOAAEEW4AOLGFHNZTTFAHM3KJZTSDBO3MFCV2OMCN6TSJO46GZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571144567 (330h16m50.03323706s) [noise=true , address=145.79.227.25:4639 , piece_id=SN5IMBBBZYXY4L24XQYZING6O2U7SBLFTPRWRU7EYAOGIGYG64IQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571284323 (330h16m50.033376817s) [noise=true , address=145.79.227.29:4944 , piece_id=6IWBVDICAIROYEKIUBJZTSIZ6XI4KJHKY24F2L7BKHWGBUQA24LA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.571824523 (330h16m50.033916996s) [noise=true , address=145.79.227.33:4900 , piece_id=5L3XYQELQU7KV7W466L5DCNXSP6U2LX7EIFOH6TUH67INZRGSWJQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.642083669 (330h16m50.104176162s) [noise=true , address=45.91.116.9:4272 , piece_id=EPDK76GOYFEJOFB4GJEB4WOKQ55L3PXKEJUQC66AUHE5BV3K3LBQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.652430721 (330h16m50.114523204s) [noise=true , address=45.91.116.12:4235 , piece_id=C3ZEP2EDRTNVHWMIH4OXWZVSACJRO3JO4FYMM5VTMGUWBFRSLRUA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.660824113 (330h16m50.122916597s) [noise=true , address=77.90.31.21:4911 , piece_id=NOFTEDILTPXQ6LHAUDHP5GKZWI6J7VXGVJZNXYPZTUKQEZ5JO4OQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.680019976 (330h16m50.142112459s) [noise=true , address=45.91.116.14:4211 , piece_id=E6CAX7G3RVE22AJT7HOESRF633GR7YDDAHJLHOSYKXTMLUIFRGZA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.705995618 (330h16m50.168088101s) [noise=true , address=77.90.31.14:4689 , piece_id=IVQS3GZNMI7JZFYVW3RHL4JSDVZ5UIHDNX5JEFV3346M3ZHQWNIQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.717852216 (330h16m50.179944699s) [noise=true , address=77.90.31.13:4643 , piece_id=UTPWYEFHE2B4BTPV7ANSJ7KUHYM4HF6NKRNPXX5NIAZQVRR3XDZQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.718777087 (330h16m50.18086957s) [noise=true , address=77.90.31.10:4526 , piece_id=IZTN2WX2Q5P2ML4EZ2OGW23HQZ3UIXBQYWLUUTXJDFU73ZUELBMQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733559335 (330h16m50.195651819s) [noise=true , address=45.91.116.5:4302 , piece_id=DDYTKB75NQGJGPKPWC6AZ4RKDU6DBLA3AX2GNKEUUWJ227IDU5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: <nil> Oct 7 07:48:04.733600604 (330h16m50.195693087s) [seg_pos={0 0}, upload=290665]: Segment reached optimal threshold of 14 pieces. Oct 7 07:48:04.734223801 (330h16m50.196316294s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1UaCjYcwJTUUrVNVJmJL4m2Gpcw9gcyGLDHySu1e3hmSkXYbtX): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734238578 (330h16m50.196331061s) [noise=true , address=77.90.31.16:4756 , piece_id=4WCDI36U4YAC3C6KLZUHS44YRYIJ6BNDCO5VPP35JYKD4WN6XQAQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734357075 (330h16m50.196449568s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12uaTGySSt8BWcdzfvw6aj2VrSvGG9xxMvTVvzn9fFA73AvxrxA): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734379557 (330h16m50.19647206s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12a7wpA6YchhigZCKC68YJzkBKwbqg1R1gBdfN8GdvrVregrK4m): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734393564 (330h16m50.196486047s) [noise=true , address=77.90.31.14:4684 , piece_id=6VCBHVVFPETZFSFBXLBCTVPLXL3AZXSFRNOFWJTCYNW75C4F2WBA, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734403833 (330h16m50.196496316s) [noise=true , address=45.91.116.6:4995 , piece_id=3IJGK3G2ZYWWRMDWOVXCNCEMAWP5YQXPNIBP5BM2LPV6GVL2H6NQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734509925 (330h16m50.196602418s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:124x1cbCPJRK8YkgSP6UzHymoLu1E1znnfg5WBiD89kSDLi8JeL): protocol: expected piece hash; context canceled; EOF Oct 7 07:48:04.734520886 (330h16m50.196613359s) [noise=true , address=77.90.31.19:4863 , piece_id=2NVNF4CV7NQVKZQZ3QOUPSCNPETR4ZKHJRBVW6TJ4RAHMZBXH5CQ, seg_pos={0 0}, upload=290665]: Long tail cancellation due to optimal threshold Oct 7 07:48:04.734627689 (330h16m50.196720172s) [seg_pos={0 0}, upload=290665]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Oct 7 07:48:04.734735696 (330h16m50.196828179s) [upload=290665]: Flush batch item: pb.BatchRequestItem_SegmentCommit Oct 7 07:48:04.734742689 (330h16m50.196835172s) [upload=290665]: Flush batch item: pb.BatchRequestItem_ObjectCommit Oct 7 07:49:01.566704025 (330h17m47.028796508s) [upload=290665]: Done uploading segments... Oct 7 07:49:01.566733461 (330h17m47.028825944s) [upload=290665]: Upload finished. err: <nil> Duration: 7.950481231s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.091029297s FAIL (2025-10-07 08:27:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.091053313s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.679694065s FAIL (2025-10-07 08:26:49 UTC) orbiter warning: operation: Commit uplink: metaclient: context deadline exceeded Duration: 5.679729391s core uplink us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.remote)