Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................X.X..........................] duration: 6.499429367s FAIL (2025-09-29 22:01:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.499466719s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................X.X.........................] duration: 6.630441307s FAIL (2025-09-29 22:00:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.630478218s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................X.X........................] duration: 7.842255381s FAIL (2025-09-29 21:59:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.842283905s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................X.X.......................] duration: 6.538361881s FAIL (2025-09-29 21:59:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.538393822s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................X.X......................] duration: 7.916525348s FAIL (2025-09-29 21:57:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.916556076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................X.X.....................] duration: 6.602057782s FAIL (2025-09-29 21:57:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.602087809s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................X.X....................] duration: 6.722013789s FAIL (2025-09-29 21:56:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.722043817s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................X.X...................] duration: 6.906233777s FAIL (2025-09-29 21:55:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.906264667s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................X.X..................] duration: 6.593256283s FAIL (2025-09-29 21:55:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.593304755s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................X.X.................] duration: 6.350849293s FAIL (2025-09-29 21:54:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.350891774s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................X.X................] duration: 6.424477895s FAIL (2025-09-29 21:54:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.424513594s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................X.X...............] duration: 8.239869542s FAIL (2025-09-29 21:53:36 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 8.239906373s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................X.X..............] duration: 6.896188386s FAIL (2025-09-29 21:52:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.896222401s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................X.X.............] duration: 6.382772586s FAIL (2025-09-29 21:52:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.382806691s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................X.X............] duration: 6.885579866s FAIL (2025-09-29 21:51:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.885609192s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................X.X...........] duration: 7.314996807s FAIL (2025-09-29 21:51:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.315033267s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................X.X..........] duration: 7.068020395s FAIL (2025-09-29 21:50:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.068048919s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................X.X.........] duration: 6.490562698s FAIL (2025-09-29 21:49:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.490593127s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................X.X........] duration: 9.874709535s FAIL (2025-09-29 21:49:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 9.874739563s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................X.X.......] duration: 6.465621439s FAIL (2025-09-29 21:48:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 6.46565282s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................X.X......] duration: 11.432789289s FAIL (2025-09-29 21:48:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 11.432823354s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................X.X.....] duration: 15.074115021s FAIL (2025-09-29 21:47:06 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 15.074139837s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................X.X....] duration: 7.071369109s FAIL (2025-09-29 21:46:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 7.071391802s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................X.X...] duration: 29.940698293s FAIL (2025-09-29 21:45:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 29.940741716s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................X.X..] duration: 26.926095538s FAIL (2025-09-29 21:45:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 26.926121797s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................X.X.] duration: 15.396866146s FAIL (2025-09-29 21:44:24 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 15.396886725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................X.X] duration: 30.283105695s FAIL (2025-09-29 21:44:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 29 21:43:38.878912975 (152h12m24.341005448s) [upload=133866]: Starting upload Sep 29 21:43:38.879000633 (152h12m24.341093116s) [upload=133866]: Done starting upload Sep 29 21:43:38.879105603 (152h12m24.341198086s) [upload=133866]: Uploading segments... Sep 29 21:43:38.879265908 (152h12m24.341358381s) [upload=133866]: Got next segment. Inline: false Sep 29 21:43:38.879325341 (152h12m24.341417824s) [upload=133866]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:43:38.879331011 (152h12m24.341423494s) [upload=133866]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:43:39.133077693 (152h12m24.595170166s) [seg_pos={0 0}, upload=133866]: Begin upload segment... Sep 29 21:43:39.133135955 (152h12m24.595228428s) [seg_pos={0 0}, upload=133866]: Done begin upload segment. Sep 29 21:43:39.133159819 (152h12m24.595252303s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133168977 (152h12m24.595261451s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133642719 (152h12m24.595735192s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.133838723 (152h12m24.595931196s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134193438 (152h12m24.596285911s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134514990 (152h12m24.596607463s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.134861730 (152h12m24.596954203s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.135190848 (152h12m24.597283321s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139069853 (152h12m24.601162326s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.139303919 (152h12m24.601396402s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140623162 (152h12m24.602715635s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.140675100 (152h12m24.602767573s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.141927174 (152h12m24.604019647s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.142481430 (152h12m24.604573913s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.144392429 (152h12m24.606484902s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:39.145672907 (152h12m24.60776539s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Uploading piece... Sep 29 21:43:41.075356652 (152h12m26.537449125s) [upload=133866]: Next returned nil segment Sep 29 21:43:41.075377122 (152h12m26.537469595s) [upload=133866]: Waiting for error group managing segments... Sep 29 21:43:41.607216833 (152h12m27.069309306s) [noise=true , address=45.91.116.11:4117 , piece_id=J2QGAAM5XPWI7PSDX7IKDZPHHWYX5VF42SNCQ5D5IAYU45VAPGQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.672327085 (152h12m27.134419558s) [noise=true , address=45.91.116.5:4299 , piece_id=AHAYD4LB54KAPHQBZZS7IZWQW73NODDU77FARZFPUSFDK425YWNA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.860185563 (152h12m27.322278046s) [noise=true , address=77.90.31.12:4589 , piece_id=B52Z7BRPJQNDIDDGWCEBEN4UVZS2UPXXBDI7UDVQEVOTVLMEI3QQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.886165734 (152h12m27.348258217s) [noise=true , address=45.91.116.8:4045 , piece_id=CVXEPONFUEAIYB77K2QDEGKZDO4ECKC4HRM7477XAL465KD2AINA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:41.977450468 (152h12m27.439542951s) [noise=true , address=45.91.116.4:4940 , piece_id=ZNXBJB73I4DA5EROXERPRX45Z2X2N6Y3V5YKZUD5UN3LDUHSFAQQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.145858296 (152h12m27.607950769s) [noise=true , address=45.91.116.6:4974 , piece_id=UP3HJFSRNVB2N7NO4EAOI4XMC5JSDF6H2GLX4O7RYJ2HCQZCCOFQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:42.819634763 (152h12m28.281727236s) [noise=true , address=145.79.227.5:4221 , piece_id=AXJDRC7ZXGWOTDOPIRALS4QYFVBBXQYDXZPBGVUE7L7KFONCGYCQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:46.238200531 (152h12m31.700293005s) [noise=true , address=145.79.227.16:4911 , piece_id=6UXOXE3FGCM7SEVIDIXNIRPUBP2ES4Z7PWTRBLQEJ7TOB32W56PA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:43:48.721473255 (152h12m34.183565738s) [noise=true , address=145.79.227.17:4969 , piece_id=24GY5KFWBW6DQYYOSNGTEOLRO7KVQTW5U6PWYY5XZDOLEQY6ZNGQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:03.803664937 (152h12m49.26575742s) [noise=true , address=145.79.227.15:4874 , piece_id=456ZKFCQOZYERDWMUHIOG4LXSHTMCM4FXXATVVLKC5ADVEJEG7DA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:05.424693268 (152h12m50.886785751s) [noise=true , address=145.79.227.4:4187 , piece_id=KQRWUHG5XNA3L5DHAIBL4QQGUCBEXND754ATRQKBISIONUOEEPWA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:06.587827241 (152h12m52.049919724s) [noise=true , address=145.79.227.14:4838 , piece_id=DTQ4LETREHJEJOP5KJ36BQDETZ4B7YNH6Z7AA33JLTCZX7BQDCBQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: <nil> Sep 29 21:44:08.179031831 (152h12m53.641124304s) [noise=true , address=145.79.227.19:4079 , piece_id=NUNYJ7OJXYVVLJQWZIG546ES4HBD6CCJURCSENMELTSTKPUMY33A, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179075023 (152h12m53.641167506s) [noise=true , address=145.79.227.18:4005 , piece_id=5V4M6Z244ZLYCPXTQREGFU775OUFCA2PE4IGL3RG6GT6BOSV4EYA, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:145.79.227.18:4005): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179082648 (152h12m53.641175121s) [noise=true , address=145.79.227.13:4803 , piece_id=Z3RA7DDGGIRZCBP6FJDZHAYG4XWXUT4PBPE2BBW3AJHKIHIUOZIQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:12HWjkru3nFUa9hvA27sHqLEmRHSyXTV45phS5kMHKWM163M4H7, address:145.79.227.13:4803): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179097807 (152h12m53.64119028s) [noise=true , address=145.79.227.12:4765 , piece_id=M7KYCIFA5M3TFLK6AUMVJSK5IW5XJCKIC4VLFYVGBLANYXTIAYDQ, seg_pos={0 0}, upload=133866]: Done uploading piece. err: ecclient: upload failed (node:1b1zhuhfXoTexYZUxKUAdCNhWJD8oGC17uXKRykA3gErEEMPWV, address:145.79.227.12:4765): protocol: expected piece hash; context deadline exceeded; EOF Sep 29 21:44:08.179124988 (152h12m53.641217471s) [seg_pos={0 0}, upload=133866]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 29 21:44:08.460340616 (152h12m53.922433089s) [upload=133866]: Done uploading segments... Sep 29 21:44:08.460374701 (152h12m53.922467184s) [upload=133866]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.283131465s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 7.442737848s FAIL (2025-09-29 21:42:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== =133831]: Uploading segments... Sep 29 21:41:17.344934347 (152h10m2.807026831s) [upload=133831]: Got next segment. Inline: false Sep 29 21:41:17.344970446 (152h10m2.807062919s) [upload=133831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:41:17.344975666 (152h10m2.807068149s) [upload=133831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:41:17.708535801 (152h10m3.170628284s) [seg_pos={0 0}, upload=133831]: Begin upload segment... Sep 29 21:41:17.708630352 (152h10m3.170722835s) [seg_pos={0 0}, upload=133831]: Done begin upload segment. Sep 29 21:41:17.708669626 (152h10m3.170762099s) [noise=true , address=145.79.227.18:4006 , piece_id=TV4XSXO7IBHIGUIGKQSTYLNFVEDGXY4Y2EO5SE4ZM7L6SAVMQGSA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.709035634 (152h10m3.171128107s) [noise=true , address=45.91.116.13:4155 , piece_id=N74LBFDCR73TJOHHGGIAMI6TTT2LC62LOH6CJIGXTNKCN4UI2QJA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.709636397 (152h10m3.17172887s) [noise=true , address=145.79.227.2:4115 , piece_id=YYF6WGK7V6ALHKBJDIQFWWPSIHJDO42MU6HKCR3YVJPKOVG2WMTA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.710023635 (152h10m3.172116108s) [noise=true , address=45.91.116.4:4937 , piece_id=BTJAJI7JMYHCOMKQZRLQU3MJI4EINY5VSVWYHIY4VUTSQPZKTAWA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.710349334 (152h10m3.172441818s) [noise=true , address=77.90.31.13:4628 , piece_id=PI5SBWVXDJF47OG6CRGHR7NAQTL5OLR6NRXWROCX65Y2KLCA3CTQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.710711595 (152h10m3.172804068s) [noise=true , address=45.91.116.10:4083 , piece_id=ZNR6E2G3KIVZVDH6ZF4C2CBEFUO273SP3IVIIC7DBSV573SF364A, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.711158546 (152h10m3.173251029s) [noise=true , address=77.90.31.17:4772 , piece_id=IBN6RCLYZTUHGCYXTPOVGPLD6PVXJKIGVZPMBMZXU6GGQUTHXCZA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.711492722 (152h10m3.173585205s) [noise=true , address=77.90.31.18:4806 , piece_id=BCKQWBKWC6QCQI4YS52LKPNGPGFAYNQHKNEPFQQYZGP35QSCAEFA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.713121123 (152h10m3.175213596s) [noise=true , address=77.90.31.11:4555 , piece_id=22GKUWH2DV3GH72OZKS6UA2QU2F24BHRHNDFTRAN7VXZBBRKUMQQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.714772236 (152h10m3.176864709s) [noise=true , address=145.79.227.7:4293 , piece_id=QHJEOYURKUEBZRTI2HW5CEJWB2Q5GEKTDNFHHQS6WSK7WWJG7MKQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.716272974 (152h10m3.178365457s) [noise=true , address=45.91.116.14:4191 , piece_id=KNCCM2RPKJULDGZOWI6RAVVNDX5RKMPK4SAEXIQVPHQGOR7EST5Q, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.717711504 (152h10m3.179803987s) [noise=true , address=145.79.227.14:4837 , piece_id=I3DCEOWA66G2V542JXYLLZG5X6IYGNEBYJYR2DMCXAVWKUTHX44Q, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.719597675 (152h10m3.181690148s) [noise=true , address=45.91.116.10:4081 , piece_id=FQZC3MIBOE7DTF7RSXSEQA3K4UMQUEZ2BTV2PDXTTELF6F5O2UMQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.721635245 (152h10m3.183727718s) [noise=true , address=77.90.31.14:4662 , piece_id=E4TIUVDYL6DNLCH7PLVQ4EOBWTHKXIY7GU4OUR5RNVLPUTPIZTIA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.723613452 (152h10m3.185705935s) [noise=true , address=145.79.227.3:4149 , piece_id=3YV63NUMKBO5XUXQVVLGS3LA3EPF54QLDKGX6XLCBHL6EFZ3KCVA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.725621466 (152h10m3.187713949s) [noise=true , address=77.90.31.12:4590 , piece_id=WJWNU7SSP5V7OPB7MBEVJZCRQOQ3CDEIS4LP2WGHZSP2SDSBEUSQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:19.650599633 (152h10m5.112692116s) [upload=133831]: Next returned nil segment Sep 29 21:41:19.650628728 (152h10m5.112721201s) [upload=133831]: Waiting for error group managing segments... Sep 29 21:41:20.164866953 (152h10m5.626959436s) [noise=true , address=45.91.116.13:4155 , piece_id=N74LBFDCR73TJOHHGGIAMI6TTT2LC62LOH6CJIGXTNKCN4UI2QJA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.249345696 (152h10m5.711438169s) [noise=true , address=77.90.31.13:4628 , piece_id=PI5SBWVXDJF47OG6CRGHR7NAQTL5OLR6NRXWROCX65Y2KLCA3CTQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.308605778 (152h10m5.770698251s) [noise=true , address=77.90.31.14:4662 , piece_id=E4TIUVDYL6DNLCH7PLVQ4EOBWTHKXIY7GU4OUR5RNVLPUTPIZTIA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.322204182 (152h10m5.784296655s) [noise=true , address=77.90.31.11:4555 , piece_id=22GKUWH2DV3GH72OZKS6UA2QU2F24BHRHNDFTRAN7VXZBBRKUMQQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.414681898 (152h10m5.876774381s) [noise=true , address=45.91.116.14:4191 , piece_id=KNCCM2RPKJULDGZOWI6RAVVNDX5RKMPK4SAEXIQVPHQGOR7EST5Q, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.458027586 (152h10m5.920120059s) [noise=true , address=77.90.31.18:4806 , piece_id=BCKQWBKWC6QCQI4YS52LKPNGPGFAYNQHKNEPFQQYZGP35QSCAEFA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.463509105 (152h10m5.925601588s) [noise=true , address=45.91.116.10:4081 , piece_id=FQZC3MIBOE7DTF7RSXSEQA3K4UMQUEZ2BTV2PDXTTELF6F5O2UMQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.465609424 (152h10m5.927701907s) [noise=true , address=77.90.31.17:4772 , piece_id=IBN6RCLYZTUHGCYXTPOVGPLD6PVXJKIGVZPMBMZXU6GGQUTHXCZA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.539763547 (152h10m6.00185603s) [noise=true , address=45.91.116.10:4083 , piece_id=ZNR6E2G3KIVZVDH6ZF4C2CBEFUO273SP3IVIIC7DBSV573SF364A, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:21.079719753 (152h10m6.541812236s) [noise=true , address=45.91.116.4:4937 , piece_id=BTJAJI7JMYHCOMKQZRLQU3MJI4EINY5VSVWYHIY4VUTSQPZKTAWA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:21.801893318 (152h10m7.263985801s) [noise=true , address=77.90.31.12:4590 , piece_id=WJWNU7SSP5V7OPB7MBEVJZCRQOQ3CDEIS4LP2WGHZSP2SDSBEUSQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:46.644333270 (152h10m32.106425753s) [noise=true , address=145.79.227.14:4837 , piece_id=I3DCEOWA66G2V542JXYLLZG5X6IYGNEBYJYR2DMCXAVWKUTHX44Q, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:42028->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:42028->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42028->145.79.227.14:4837: use of closed network connection Sep 29 21:41:46.644461234 (152h10m32.106553717s) [noise=true , address=145.79.227.18:4006 , piece_id=TV4XSXO7IBHIGUIGKQSTYLNFVEDGXY4Y2EO5SE4ZM7L6SAVMQGSA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:57026->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:57026->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57026->145.79.227.18:4006: use of closed network connection Sep 29 21:41:46.644553449 (152h10m32.106645922s) [noise=true , address=145.79.227.3:4149 , piece_id=3YV63NUMKBO5XUXQVVLGS3LA3EPF54QLDKGX6XLCBHL6EFZ3KCVA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:37738->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:37738->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37738->145.79.227.3:4149: use of closed network connection Sep 29 21:41:46.644652388 (152h10m32.106744861s) [noise=true , address=145.79.227.2:4115 , piece_id=YYF6WGK7V6ALHKBJDIQFWWPSIHJDO42MU6HKCR3YVJPKOVG2WMTA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:59644->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:59644->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59644->145.79.227.2:4115: use of closed network connection Sep 29 21:41:46.644747008 (152h10m32.106839481s) [noise=true , address=145.79.227.7:4293 , piece_id=QHJEOYURKUEBZRTI2HW5CEJWB2Q5GEKTDNFHHQS6WSK7WWJG7MKQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:45554->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:45554->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45554->145.79.227.7:4293: use of closed network connection Sep 29 21:41:46.644771625 (152h10m32.106864109s) [seg_pos={0 0}, upload=133831]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 29 21:41:46.902201919 (152h10m32.364294422s) [upload=133831]: Done uploading segments... Sep 29 21:41:46.902239782 (152h10m32.364332255s) [upload=133831]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 7.442777053s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.259894043s FAIL (2025-09-29 21:41:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== =133831]: Uploading segments... Sep 29 21:41:17.344934347 (152h10m2.807026831s) [upload=133831]: Got next segment. Inline: false Sep 29 21:41:17.344970446 (152h10m2.807062919s) [upload=133831]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 29 21:41:17.344975666 (152h10m2.807068149s) [upload=133831]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 29 21:41:17.708535801 (152h10m3.170628284s) [seg_pos={0 0}, upload=133831]: Begin upload segment... Sep 29 21:41:17.708630352 (152h10m3.170722835s) [seg_pos={0 0}, upload=133831]: Done begin upload segment. Sep 29 21:41:17.708669626 (152h10m3.170762099s) [noise=true , address=145.79.227.18:4006 , piece_id=TV4XSXO7IBHIGUIGKQSTYLNFVEDGXY4Y2EO5SE4ZM7L6SAVMQGSA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.709035634 (152h10m3.171128107s) [noise=true , address=45.91.116.13:4155 , piece_id=N74LBFDCR73TJOHHGGIAMI6TTT2LC62LOH6CJIGXTNKCN4UI2QJA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.709636397 (152h10m3.17172887s) [noise=true , address=145.79.227.2:4115 , piece_id=YYF6WGK7V6ALHKBJDIQFWWPSIHJDO42MU6HKCR3YVJPKOVG2WMTA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.710023635 (152h10m3.172116108s) [noise=true , address=45.91.116.4:4937 , piece_id=BTJAJI7JMYHCOMKQZRLQU3MJI4EINY5VSVWYHIY4VUTSQPZKTAWA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.710349334 (152h10m3.172441818s) [noise=true , address=77.90.31.13:4628 , piece_id=PI5SBWVXDJF47OG6CRGHR7NAQTL5OLR6NRXWROCX65Y2KLCA3CTQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.710711595 (152h10m3.172804068s) [noise=true , address=45.91.116.10:4083 , piece_id=ZNR6E2G3KIVZVDH6ZF4C2CBEFUO273SP3IVIIC7DBSV573SF364A, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.711158546 (152h10m3.173251029s) [noise=true , address=77.90.31.17:4772 , piece_id=IBN6RCLYZTUHGCYXTPOVGPLD6PVXJKIGVZPMBMZXU6GGQUTHXCZA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.711492722 (152h10m3.173585205s) [noise=true , address=77.90.31.18:4806 , piece_id=BCKQWBKWC6QCQI4YS52LKPNGPGFAYNQHKNEPFQQYZGP35QSCAEFA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.713121123 (152h10m3.175213596s) [noise=true , address=77.90.31.11:4555 , piece_id=22GKUWH2DV3GH72OZKS6UA2QU2F24BHRHNDFTRAN7VXZBBRKUMQQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.714772236 (152h10m3.176864709s) [noise=true , address=145.79.227.7:4293 , piece_id=QHJEOYURKUEBZRTI2HW5CEJWB2Q5GEKTDNFHHQS6WSK7WWJG7MKQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.716272974 (152h10m3.178365457s) [noise=true , address=45.91.116.14:4191 , piece_id=KNCCM2RPKJULDGZOWI6RAVVNDX5RKMPK4SAEXIQVPHQGOR7EST5Q, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.717711504 (152h10m3.179803987s) [noise=true , address=145.79.227.14:4837 , piece_id=I3DCEOWA66G2V542JXYLLZG5X6IYGNEBYJYR2DMCXAVWKUTHX44Q, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.719597675 (152h10m3.181690148s) [noise=true , address=45.91.116.10:4081 , piece_id=FQZC3MIBOE7DTF7RSXSEQA3K4UMQUEZ2BTV2PDXTTELF6F5O2UMQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.721635245 (152h10m3.183727718s) [noise=true , address=77.90.31.14:4662 , piece_id=E4TIUVDYL6DNLCH7PLVQ4EOBWTHKXIY7GU4OUR5RNVLPUTPIZTIA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.723613452 (152h10m3.185705935s) [noise=true , address=145.79.227.3:4149 , piece_id=3YV63NUMKBO5XUXQVVLGS3LA3EPF54QLDKGX6XLCBHL6EFZ3KCVA, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:17.725621466 (152h10m3.187713949s) [noise=true , address=77.90.31.12:4590 , piece_id=WJWNU7SSP5V7OPB7MBEVJZCRQOQ3CDEIS4LP2WGHZSP2SDSBEUSQ, seg_pos={0 0}, upload=133831]: Uploading piece... Sep 29 21:41:19.650599633 (152h10m5.112692116s) [upload=133831]: Next returned nil segment Sep 29 21:41:19.650628728 (152h10m5.112721201s) [upload=133831]: Waiting for error group managing segments... Sep 29 21:41:20.164866953 (152h10m5.626959436s) [noise=true , address=45.91.116.13:4155 , piece_id=N74LBFDCR73TJOHHGGIAMI6TTT2LC62LOH6CJIGXTNKCN4UI2QJA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.249345696 (152h10m5.711438169s) [noise=true , address=77.90.31.13:4628 , piece_id=PI5SBWVXDJF47OG6CRGHR7NAQTL5OLR6NRXWROCX65Y2KLCA3CTQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.308605778 (152h10m5.770698251s) [noise=true , address=77.90.31.14:4662 , piece_id=E4TIUVDYL6DNLCH7PLVQ4EOBWTHKXIY7GU4OUR5RNVLPUTPIZTIA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.322204182 (152h10m5.784296655s) [noise=true , address=77.90.31.11:4555 , piece_id=22GKUWH2DV3GH72OZKS6UA2QU2F24BHRHNDFTRAN7VXZBBRKUMQQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.414681898 (152h10m5.876774381s) [noise=true , address=45.91.116.14:4191 , piece_id=KNCCM2RPKJULDGZOWI6RAVVNDX5RKMPK4SAEXIQVPHQGOR7EST5Q, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.458027586 (152h10m5.920120059s) [noise=true , address=77.90.31.18:4806 , piece_id=BCKQWBKWC6QCQI4YS52LKPNGPGFAYNQHKNEPFQQYZGP35QSCAEFA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.463509105 (152h10m5.925601588s) [noise=true , address=45.91.116.10:4081 , piece_id=FQZC3MIBOE7DTF7RSXSEQA3K4UMQUEZ2BTV2PDXTTELF6F5O2UMQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.465609424 (152h10m5.927701907s) [noise=true , address=77.90.31.17:4772 , piece_id=IBN6RCLYZTUHGCYXTPOVGPLD6PVXJKIGVZPMBMZXU6GGQUTHXCZA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:20.539763547 (152h10m6.00185603s) [noise=true , address=45.91.116.10:4083 , piece_id=ZNR6E2G3KIVZVDH6ZF4C2CBEFUO273SP3IVIIC7DBSV573SF364A, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:21.079719753 (152h10m6.541812236s) [noise=true , address=45.91.116.4:4937 , piece_id=BTJAJI7JMYHCOMKQZRLQU3MJI4EINY5VSVWYHIY4VUTSQPZKTAWA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:21.801893318 (152h10m7.263985801s) [noise=true , address=77.90.31.12:4590 , piece_id=WJWNU7SSP5V7OPB7MBEVJZCRQOQ3CDEIS4LP2WGHZSP2SDSBEUSQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: <nil> Sep 29 21:41:46.644333270 (152h10m32.106425753s) [noise=true , address=145.79.227.14:4837 , piece_id=I3DCEOWA66G2V542JXYLLZG5X6IYGNEBYJYR2DMCXAVWKUTHX44Q, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12rJPJdZ2EA1Uh9gUa7dHj5utLBtcFQYFPXPVX5HgJbo8ppc6pV, address:145.79.227.14:4837): protocol: write tcp 188.166.225.170:42028->145.79.227.14:4837: use of closed network connection; write tcp 188.166.225.170:42028->145.79.227.14:4837: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42028->145.79.227.14:4837: use of closed network connection Sep 29 21:41:46.644461234 (152h10m32.106553717s) [noise=true , address=145.79.227.18:4006 , piece_id=TV4XSXO7IBHIGUIGKQSTYLNFVEDGXY4Y2EO5SE4ZM7L6SAVMQGSA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:1ig3zGgKZUH8jwXEojp2A6tYGVFCL33PUDkhAbEjS6affBLtmc, address:145.79.227.18:4006): protocol: write tcp 188.166.225.170:57026->145.79.227.18:4006: use of closed network connection; write tcp 188.166.225.170:57026->145.79.227.18:4006: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:57026->145.79.227.18:4006: use of closed network connection Sep 29 21:41:46.644553449 (152h10m32.106645922s) [noise=true , address=145.79.227.3:4149 , piece_id=3YV63NUMKBO5XUXQVVLGS3LA3EPF54QLDKGX6XLCBHL6EFZ3KCVA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm, address:145.79.227.3:4149): protocol: write tcp 188.166.225.170:37738->145.79.227.3:4149: use of closed network connection; write tcp 188.166.225.170:37738->145.79.227.3:4149: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:37738->145.79.227.3:4149: use of closed network connection Sep 29 21:41:46.644652388 (152h10m32.106744861s) [noise=true , address=145.79.227.2:4115 , piece_id=YYF6WGK7V6ALHKBJDIQFWWPSIHJDO42MU6HKCR3YVJPKOVG2WMTA, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12DKJTyYZ5om3b3NgCn2HXBwGbN3aoi2fk6gb9H5gAbVbHFaeoR, address:145.79.227.2:4115): protocol: write tcp 188.166.225.170:59644->145.79.227.2:4115: use of closed network connection; write tcp 188.166.225.170:59644->145.79.227.2:4115: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59644->145.79.227.2:4115: use of closed network connection Sep 29 21:41:46.644747008 (152h10m32.106839481s) [noise=true , address=145.79.227.7:4293 , piece_id=QHJEOYURKUEBZRTI2HW5CEJWB2Q5GEKTDNFHHQS6WSK7WWJG7MKQ, seg_pos={0 0}, upload=133831]: Done uploading piece. err: ecclient: upload failed (node:12jsNwKfNEjPG9fmLdvdJwMYPHeh5w3KtWA1MsXZ9Ns8QBpVYbE, address:145.79.227.7:4293): protocol: write tcp 188.166.225.170:45554->145.79.227.7:4293: use of closed network connection; write tcp 188.166.225.170:45554->145.79.227.7:4293: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:45554->145.79.227.7:4293: use of closed network connection Sep 29 21:41:46.644771625 (152h10m32.106864109s) [seg_pos={0 0}, upload=133831]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 29 21:41:46.902201919 (152h10m32.364294422s) [upload=133831]: Done uploading segments... Sep 29 21:41:46.902239782 (152h10m32.364332255s) [upload=133831]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.259941735s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.919402859s FAIL (2025-09-29 15:34:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.919433137s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.809920726s FAIL (2025-09-29 15:34:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.809944951s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 5.91438898s FAIL (2025-09-29 15:33:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.914420089s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 5.648173159s FAIL (2025-09-29 15:32:15 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.64820587s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 5.792123064s FAIL (2025-09-29 15:31:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.792156308s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.716354448s FAIL (2025-09-29 15:30:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.716382413s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 5.677294392s FAIL (2025-09-29 15:29:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.677326803s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 5.673901852s FAIL (2025-09-29 15:28:54 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.674067105s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 5.65389231s FAIL (2025-09-29 15:28:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.654499636s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [X........X..................................................] duration: 5.852982562s FAIL (2025-09-29 15:27:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.853003332s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.X........X.................................................] duration: 5.724639804s FAIL (2025-09-29 15:27:08 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.724672917s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..X........X................................................] duration: 5.838342093s FAIL (2025-09-29 15:26:21 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.838573755s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...X........X...............................................] duration: 5.611131075s FAIL (2025-09-29 15:25:22 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.611180349s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....X........X..............................................] duration: 5.701244444s FAIL (2025-09-29 15:24:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.701272248s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....X........X.............................................] duration: 5.439396916s FAIL (2025-09-29 15:24:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.439429177s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [......X........X............................................] duration: 5.762594188s FAIL (2025-09-29 15:23:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.762622693s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.......X........X...........................................] duration: 5.825623806s FAIL (2025-09-29 15:22:07 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.825655768s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [........X........X..........................................] duration: 5.679281104s FAIL (2025-09-29 15:21:01 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.679315599s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.........X........X.........................................] duration: 5.857920956s FAIL (2025-09-29 15:20:26 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.857949721s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..........X........X........................................] duration: 5.751277247s FAIL (2025-09-29 15:19:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.751302697s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...........X........X.......................................] duration: 5.616330004s FAIL (2025-09-29 15:19:00 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.616357265s core uplink us-select-1 us1-placement-20

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)