Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 9.87452536s FAIL (2025-09-28 06:14:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 9.874554816s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 6.101175205s FAIL (2025-09-28 06:14:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.101216255s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 6.145266512s FAIL (2025-09-28 06:12:59 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.145293194s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 6.272780261s FAIL (2025-09-28 06:12:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.272812191s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 6.342689111s FAIL (2025-09-28 06:11:05 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.342736441s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 6.18318141s FAIL (2025-09-28 06:10:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.183216776s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 6.341086136s FAIL (2025-09-28 06:09:07 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.341121843s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 6.290631742s FAIL (2025-09-28 06:08:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.290665797s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 6.417790044s FAIL (2025-09-28 06:07:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.417826383s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 6.453371305s FAIL (2025-09-28 06:06:34 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.453407725s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 6.20718196s FAIL (2025-09-28 06:05:20 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.207210543s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................X........................] duration: 6.21597186s FAIL (2025-09-28 06:05:02 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.216013016s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................X.......................] duration: 6.006351855s FAIL (2025-09-28 06:04:40 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.00638031s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................X......................] duration: 6.192630662s FAIL (2025-09-28 06:03:30 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.192668966s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................X.....................] duration: 6.055447072s FAIL (2025-09-28 06:02:52 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.055518076s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................X....................] duration: 6.504244565s FAIL (2025-09-28 06:02:04 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.504300992s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................X...................] duration: 6.258799837s FAIL (2025-09-28 06:01:03 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.258836918s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................X..................] duration: 6.045234667s FAIL (2025-09-28 06:00:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.04527811s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................X.................] duration: 6.044475277s FAIL (2025-09-28 05:59:29 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.044506718s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................X................] duration: 6.102186921s FAIL (2025-09-28 05:58:27 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.102216738s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............................................X...............] duration: 6.128358975s FAIL (2025-09-28 05:58:08 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.128422247s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............................................X..............] duration: 6.173376227s FAIL (2025-09-28 05:57:26 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.173407859s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..............................................X.............] duration: 6.057080592s FAIL (2025-09-28 05:56:50 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.057140405s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...............................................X............] duration: 6.404305466s FAIL (2025-09-28 05:55:42 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.404337287s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [................................................X...........] duration: 6.539625427s FAIL (2025-09-28 05:55:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.539691995s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.................................................X..........] duration: 6.149886228s FAIL (2025-09-28 05:54:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.149919682s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..................................................X.........] duration: 6.168613958s FAIL (2025-09-28 05:53:57 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.168645757s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...................................................X........] duration: 6.139278732s FAIL (2025-09-28 05:53:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.139317677s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....................................................X.......] duration: 6.084094762s FAIL (2025-09-28 05:52:19 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.08414601s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 6.153456884s FAIL (2025-09-28 05:51:38 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.153506409s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 6.119431196s FAIL (2025-09-28 05:50:43 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.119454219s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.186562466s FAIL (2025-09-28 05:50:23 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.186599526s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 6.135148313s FAIL (2025-09-28 05:49:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.13517892s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 6.123518168s FAIL (2025-09-28 05:48:47 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.123550499s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 6.06221635s FAIL (2025-09-28 05:47:55 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 6.062239795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 2m36.455661913s FAIL (2025-09-28 05:47:49 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: metaclient: context deadline exceeded ======LOGS====== Sep 28 05:45:13.490333278 (112h13m58.952425761s) [upload=98771]: Starting upload Sep 28 05:45:13.490430243 (112h13m58.952522726s) [upload=98771]: Done starting upload Sep 28 05:45:13.490640914 (112h13m58.952733417s) [upload=98771]: Uploading segments... Sep 28 05:45:13.490694305 (112h13m58.952786808s) [upload=98771]: Got next segment. Inline: false Sep 28 05:45:13.490731998 (112h13m58.952824481s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 28 05:45:13.490737418 (112h13m58.952829901s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 28 05:45:13.756642224 (112h13m59.218734707s) [seg_pos={0 0}, upload=98771]: Begin upload segment... Sep 28 05:45:13.756723488 (112h13m59.218815971s) [seg_pos={0 0}, upload=98771]: Done begin upload segment. Sep 28 05:45:13.756758295 (112h13m59.218850778s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.757529162 (112h13m59.219621645s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759773316 (112h13m59.221865789s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.759805997 (112h13m59.221898501s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.760535637 (112h13m59.22262812s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762028019 (112h13m59.224120502s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.762256564 (112h13m59.224349047s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.763691236 (112h13m59.225783749s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.764676171 (112h13m59.226768654s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.765267547 (112h13m59.22736003s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.766936505 (112h13m59.229028978s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.767243240 (112h13m59.229335723s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.768747884 (112h13m59.230840368s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.769344260 (112h13m59.231436743s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.770503588 (112h13m59.232596071s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:13.771352304 (112h13m59.233444787s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Uploading piece... Sep 28 05:45:15.470974411 (112h14m0.933066894s) [upload=98771]: Next returned nil segment Sep 28 05:45:15.471001192 (112h14m0.933093676s) [upload=98771]: Waiting for error group managing segments... Sep 28 05:45:15.748295997 (112h14m1.21038849s) [noise=true , address=145.79.227.3:4150 , piece_id=L5MSYDNVHQRSD3HHMSEMUQVJRDQLDC5CE6WA5PSJQ5AVNYRC6SEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.756244582 (112h14m1.218337065s) [noise=true , address=145.79.227.13:4804 , piece_id=6VPMHQSC6YUO4CBA2T52OWVJFBHPMUFWZV76QMLCPFMX3E63EI6A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.779844741 (112h14m1.241937224s) [noise=true , address=145.79.227.16:4911 , piece_id=P5RKDU74GCHJQHKGE5PGDI7FNTRU2XTBBSX43SOPDQFOQMTJKEEQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.780377786 (112h14m1.242470269s) [noise=true , address=145.79.227.7:4295 , piece_id=6IAO544IPMPVBGCZHJF64JUB57EP7SPSQRBKK6DGGRSOOMS2EDAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.849442015 (112h14m1.311534508s) [noise=true , address=45.91.116.12:4227 , piece_id=QWQS5G2DU2CNHBYMZYALAGQ6QYQWHO6TKIBNNTSUYKPGWO3HH5SA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.860055395 (112h14m1.322147888s) [noise=true , address=45.91.116.12:4225 , piece_id=SY2F7C5774NDKIQLCCP6O45O7IFHKJLYCYCIBKPPFCM6PS3K265A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.881166714 (112h14m1.343259197s) [noise=true , address=45.91.116.4:4940 , piece_id=VWAC7JMH6JNFGT2W5F5YDGQI26ZNJEV43KYW36SKOS5PQ5F3KG4Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.886334144 (112h14m1.348426627s) [noise=true , address=45.91.116.5:4299 , piece_id=J7LDC3WLHPL2VGH3GQFV7Y2HMW252BOE2JQUKNWS3HVQZDDYLVDA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.916861703 (112h14m1.378954176s) [noise=true , address=45.91.116.10:4081 , piece_id=K35XCHTLLPYJLVZUPFSXWLPZ5JFUPAFWZQQQKGHJ2ARQJMLEVKTA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.949455187 (112h14m1.41154767s) [noise=true , address=45.91.116.10:4082 , piece_id=XQXKZNJGR5MM5U45XXWZ6ZWUC2GGXAEK4CGCRZDZ2DAY6ZGKOONQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.960326568 (112h14m1.422419051s) [noise=true , address=77.90.31.10:4517 , piece_id=AVJZ2NZGUCLTE72GNODFA7PETMBNU7VQEB6KSAE2LCI5YBCID63Q, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971842776 (112h14m1.433935259s) [noise=true , address=77.90.31.15:4698 , piece_id=SJNDTFUZLIR66Q5TFVTZ5OHRATBMGU3YV6JZV2UE7UO3TNTRGQAA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971929381 (112h14m1.434021864s) [noise=true , address=77.90.31.16:4734 , piece_id=6A6PUHGDJ2BUAZNEGD7ZKUGQX5Y3VOBW7TBOPHZQBTIXWN5BUNVA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.971941263 (112h14m1.434033747s) [seg_pos={0 0}, upload=98771]: Segment reached optimal threshold of 13 pieces. Sep 28 05:45:15.972138199 (112h14m1.434230692s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ULMQbw1PRksy5un9tNpwTQFgZoNsy9PhUn86Ny9itKAkesshz): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972150642 (112h14m1.434243125s) [noise=true , address=77.90.31.11:4553 , piece_id=54PJBQ2R3YB7ZKT3D7GNR7QXYFZZSNLFUKWSKVPRWXC6KVIN2ZJA, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972268257 (112h14m1.43436073s) [noise=true , address=45.91.116.13:4153 , piece_id=CX3YAKWL2VVD3VCHGRQVT2H5QZZLNY5OZKSQ3KUCBJ2RGN35E2TQ, seg_pos={0 0}, upload=98771]: Done uploading piece. err: <nil> Sep 28 05:45:15.972322419 (112h14m1.434414912s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1288A5aF4EsQG3XrDQg6fqURsqsFKfSRMeLi4kC5aBRYF6uBnFB): protocol: expected piece hash; context canceled; EOF Sep 28 05:45:15.972331947 (112h14m1.43442443s) [noise=true , address=45.91.116.8:4048 , piece_id=ZV7KCAT4JIYA6L7CMOGEYCFNIAGDBKR3CIYWSWDNRMOR5OMFEQ7A, seg_pos={0 0}, upload=98771]: Long tail cancellation due to optimal threshold Sep 28 05:45:15.972352787 (112h14m1.43444527s) [seg_pos={0 0}, upload=98771]: Done waiting for segment. successful: 14 optimal: 13 errs: <nil> Sep 28 05:45:15.972458128 (112h14m1.434550611s) [upload=98771]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 28 05:45:15.972464690 (112h14m1.434557173s) [upload=98771]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 28 05:47:49.198854645 (112h16m34.660947178s) [upload=98771]: Done uploading segments... Sep 28 05:47:49.198881104 (112h16m34.660973587s) [upload=98771]: Upload finished. err: metaclient: context deadline exceeded Duration: 2m36.455692813s core uplink us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.147217875s FAIL (2025-09-28 05:22:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.147248543s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 6.493263306s FAIL (2025-09-28 05:21:25 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.493287142s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 6.182509928s FAIL (2025-09-28 05:20:29 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.182539454s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 6.170598414s FAIL (2025-09-28 05:19:34 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.170626877s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 6.31706505s FAIL (2025-09-28 05:19:17 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.317137067s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 5.722922941s FAIL (2025-09-28 05:18:47 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 5.722949924s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 6.549775649s FAIL (2025-09-28 05:18:18 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.54980755s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 6.550697834s FAIL (2025-09-28 05:17:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.550996223s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 6.265864502s FAIL (2025-09-28 05:17:19 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.265898096s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 6.193728065s FAIL (2025-09-28 05:16:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.193780675s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 6.475702075s FAIL (2025-09-28 05:16:04 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.475730399s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 6.290577605s FAIL (2025-09-28 05:14:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.29060642s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 6.357940337s FAIL (2025-09-28 05:13:48 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.35796872s edge gateway us-select-1
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 6.620400572s FAIL (2025-09-28 05:12:56 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.620427021s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)