Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.138461386s FAIL (2025-09-28 22:59:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.138492827s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 2.073777821s FAIL (2025-09-28 22:59:30 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.073804301s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 2.054669735s FAIL (2025-09-28 22:59:09 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.05470466s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.442131988s FAIL (2025-09-28 22:58:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.442167406s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 2.008170432s FAIL (2025-09-28 22:58:52 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.008200499s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.150315317s FAIL (2025-09-28 22:58:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.150346707s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 2.428395434s FAIL (2025-09-28 22:58:01 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.428502859s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.133016087s FAIL (2025-09-28 22:57:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.133053008s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 2.100770199s FAIL (2025-09-28 22:56:51 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.100803502s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.200653753s FAIL (2025-09-28 22:56:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.2006785s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.083442241s FAIL (2025-09-28 22:56:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.083481205s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 2.304674387s FAIL (2025-09-28 22:55:50 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.304708343s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.329805209s FAIL (2025-09-28 22:55:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.329831508s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 2.12605829s FAIL (2025-09-28 22:55:18 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.126133474s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.103190854s FAIL (2025-09-28 22:54:48 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.103223066s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 2.069999135s FAIL (2025-09-28 22:54:47 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.07002889s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.542410115s FAIL (2025-09-28 22:54:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.542431235s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 2.003312503s FAIL (2025-09-28 22:54:20 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.003343862s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.387679166s FAIL (2025-09-28 22:54:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.38782376s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.028786294s FAIL (2025-09-28 22:53:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.02881083s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 2.029546777s FAIL (2025-09-28 22:53:24 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.029654993s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.202810564s FAIL (2025-09-28 22:53:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.202833547s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 2.142603171s FAIL (2025-09-28 22:53:01 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.142654749s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.035780241s FAIL (2025-09-28 22:52:33 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.035809115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 2.130738785s FAIL (2025-09-28 22:52:30 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.130784642s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.141288356s FAIL (2025-09-28 22:52:15 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.141315076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 2.028982504s FAIL (2025-09-28 22:51:46 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.029010067s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.195354396s FAIL (2025-09-28 22:51:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.195388792s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 2.201326346s FAIL (2025-09-28 22:51:00 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 2.201351073s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 41.798479526s FAIL (2025-09-28 22:50:46 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 22:50:24.901683454 (129h19m10.363775927s) [upload=113836]: Starting upload Sep 28 22:50:24.901752295 (129h19m10.363844768s) [upload=113836]: Done starting upload Sep 28 22:50:24.901879066 (129h19m10.363971539s) [upload=113836]: Uploading segments... Sep 28 22:50:24.902030415 (129h19m10.364122898s) [upload=113836]: Got next segment. Inline: false Sep 28 22:50:24.902077033 (129h19m10.364169516s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 22:50:24.902083035 (129h19m10.364175508s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 22:50:26.852393655 (129h19m12.314486128s) [seg_pos={0 0}, upload=113836]: Begin upload segment... Sep 28 22:50:26.852460853 (129h19m12.314553326s) [seg_pos={0 0}, upload=113836]: Done begin upload segment. Sep 28 22:50:26.852475400 (129h19m12.314567873s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.852488045 (129h19m12.314580518s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853029766 (129h19m12.315122249s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853296834 (129h19m12.315389317s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853540498 (129h19m12.315632991s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853566488 (129h19m12.315658971s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853606073 (129h19m12.315698556s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.853931903 (129h19m12.316024376s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854100585 (129h19m12.316193068s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854231675 (129h19m12.316324148s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854504174 (129h19m12.316596647s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854593263 (129h19m12.316685736s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.854764679 (129h19m12.316857152s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855066444 (129h19m12.317158917s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855570193 (129h19m12.317662676s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:26.855841500 (129h19m12.317933983s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Uploading piece... Sep 28 22:50:28.582844923 (129h19m14.044937406s) [upload=113836]: Next returned nil segment Sep 28 22:50:28.582866864 (129h19m14.044959347s) [upload=113836]: Waiting for error group managing segments... Sep 28 22:50:28.881030230 (129h19m14.343122713s) [noise=true , address=145.79.227.4:4186 , piece_id=DZPVZ6TTS523WUKPQBVTDHWDLRFMRWMT673YDGKIOGD6Y6LJLLKQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.887290389 (129h19m14.349382872s) [noise=true , address=145.79.227.16:4910 , piece_id=YNU3QIF56QQQFB5UQITRXEKNDQDBO3FJ4Q4WFSEGG74FHCNNKDWQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888070155 (129h19m14.350162638s) [noise=true , address=145.79.227.7:4294 , piece_id=SSW3KWRKDWZJ2JCNATQQUSFT3YGDD2AYBRCQQ466BSIS243MX7JQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888212486 (129h19m14.350304979s) [noise=true , address=145.79.227.5:4221 , piece_id=AISUWYT5DEQNHRO4XCSCAWMRWQZJALZM34H46MHYQU6FZ366VE2Q, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.888968586 (129h19m14.351061059s) [noise=true , address=145.79.227.12:4765 , piece_id=LGXQUKQ23WX23FACNH6WXAV5X2K4PUVQFVG5OLCMZTJEFJJDAXCQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.901633701 (129h19m14.363726184s) [noise=true , address=145.79.227.6:4258 , piece_id=GW2CCZ7LS52Q4BZW6D7LMCY4IIK3P7TG5J4VPSS4CKLRCF7CSOQA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905686599 (129h19m14.367779082s) [noise=true , address=145.79.227.13:4801 , piece_id=NCEQJYSYM37IJT724RGYQZT3BEQT6EMHW2PN6YYLJHKHNDZ7IB4A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:28.905792000 (129h19m14.367884483s) [noise=true , address=145.79.227.3:4151 , piece_id=6L2HCDQHVN43MLKGSCSQDYYHOWGPURZ2AHIDPT6A577RZS22ECXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.038837341 (129h19m14.500929824s) [noise=true , address=45.91.116.7:4011 , piece_id=VRHQ7UKQYOYZH6O4IP55CN5SP6YTOIYKA3B4M3IQUAOKLQV2PRMQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.052413222 (129h19m14.514505705s) [noise=true , address=45.91.116.6:4974 , piece_id=ZSNORZRG5XAPDFYHO2NZ2EZZWU2767SUYXYBWBU77T25VLXGRXFA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.124522220 (129h19m14.586614693s) [noise=true , address=77.90.31.18:4807 , piece_id=PAE3COMCDOBJKH73EPUDMTSVPNILJH6SZ7HWXOWBJVP3CUBNXBSA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.166734780 (129h19m14.628827273s) [noise=true , address=45.91.116.5:4297 , piece_id=WOTCIC7IJ2MSJBNUIXRMC5M5BOORRVKI6DMBTKO7SMPCGP2KXNXQ, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.169989688 (129h19m14.632082172s) [noise=true , address=45.91.116.10:4081 , piece_id=FYELLZRKVZ346QV2ZZ76UB7SXS7H64THXPWSHQJHZ4HLZRJCC35A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: <nil> Sep 28 22:50:29.170018583 (129h19m14.632111066s) [seg_pos={0 0}, upload=113836]: Segment reached optimal threshold of 13 pieces. Sep 28 22:50:29.170184509 (129h19m14.632276992s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12FS8qbYudyqYzaKkQmN2sbHGBcyBCgUoVEzMmCFB2VPHeB6YQ1): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170196101 (129h19m14.632288584s) [noise=true , address=45.91.116.12:4226 , piece_id=7CKQIFZIRGMM6ZIMTUNG45GPTYBK4BV2V5E4D3PBGDXRONY6YYZA, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170339364 (129h19m14.632431847s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12WpRXtVHj1EPPPhGAgAKVCPtEXQXMESSiVHhamxpy7jgRunmFu): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170347790 (129h19m14.632440263s) [noise=true , address=77.90.31.13:4626 , piece_id=B6VZSRKLCVJ5P2KJCHH7KWWLLIPGNTVDQ7QUOTDTCDHH6Q33CP5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170461887 (129h19m14.63255437s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sFeLTeZ6ox1U2b5dDorbFq1mc5rPXdFjYgUCGFzRmRYv4oNpK): protocol: expected piece hash; context canceled; EOF Sep 28 22:50:29.170476124 (129h19m14.632568627s) [noise=true , address=77.90.31.21:4878 , piece_id=NAPYUKEGFXO5KIGC6WJI5WPJPKLHGWB365L6N4MDBYYNC7ZCDC5A, seg_pos={0 0}, upload=113836]: Long tail cancellation due to optimal threshold Sep 28 22:50:29.170500641 (129h19m14.632593114s) [seg_pos={0 0}, upload=113836]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 28 22:50:29.170600972 (129h19m14.632693445s) [upload=113836]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 22:50:29.170609267 (129h19m14.63270175s) [upload=113836]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 22:50:46.053429497 (129h19m31.51552197s) [upload=113836]: Done uploading segments... Sep 28 22:50:46.053458162 (129h19m31.515550635s) [upload=113836]: Upload finished. err: metaclient: context deadline exceeded Duration: 41.798508331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 25.889964301s FAIL (2025-09-28 22:50:38 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 25.88999075s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.00129208s FAIL (2025-09-28 22:50:08 UTC) orbiter warning: operation: DeleteObject uplink: metaclient: context deadline exceeded Duration: 30.001317468s core uplink us1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.139764393s FAIL (2025-09-28 06:32:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.139803549s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.821857273s FAIL (2025-09-28 06:32:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 5.821890787s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.34336744s FAIL (2025-09-28 06:31:13 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.34341977s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.058631582s FAIL (2025-09-28 06:30:00 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.058665448s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 7.368560287s FAIL (2025-09-28 06:29:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 7.36859363s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 6.095576964s FAIL (2025-09-28 06:28:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.09561194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.133446754s FAIL (2025-09-28 06:27:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.133476742s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.170308715s FAIL (2025-09-28 06:27:01 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.170339624s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.121546076s FAIL (2025-09-28 06:26:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.121580794s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.187223968s FAIL (2025-09-28 06:25:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.187254326s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.235440771s FAIL (2025-09-28 06:24:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.235473543s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.001999658s FAIL (2025-09-28 06:24:21 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.002032972s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.161871201s FAIL (2025-09-28 06:23:37 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.161915255s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.651435519s FAIL (2025-09-28 06:22:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.651483662s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 6.522611305s FAIL (2025-09-28 06:22:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.522646802s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 6.057551639s FAIL (2025-09-28 06:21:44 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.057582898s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 6.060820482s FAIL (2025-09-28 06:21:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.060853476s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 6.683778296s FAIL (2025-09-28 06:20:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.683812552s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)