Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.....................................................X......] duration: 5.597361448s FAIL (2025-09-24 16:19:28 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.597389881s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [......................................................X.....] duration: 5.641528285s FAIL (2025-09-24 16:18:28 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.641561279s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.......................................................X....] duration: 6.735819658s FAIL (2025-09-24 16:17:42 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 6.735837793s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [........................................................X...] duration: 5.891787125s FAIL (2025-09-24 16:17:17 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.891823874s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [.........................................................X..] duration: 5.830330814s FAIL (2025-09-24 16:17:01 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.830383444s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [..........................................................X.] duration: 5.80592544s FAIL (2025-09-24 16:16:39 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 5.805950016s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 1m0.000384683s FAIL (2025-09-24 16:16:33 UTC) orbiter warning: operation: StatBucket ======ERROR====== uplink: bucket: metaclient: context deadline exceeded ======LOGS====== Duration: 1m0.000409722s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 11.713752651s FAIL (2025-09-24 14:54:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 11.713780795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 7.24552949s FAIL (2025-09-24 14:53:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.245554468s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 7.687833313s FAIL (2025-09-24 14:52:37 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.687855807s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 7.298684267s FAIL (2025-09-24 14:51:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.298719795s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXXX.......................................................] duration: 7.259727118s FAIL (2025-09-24 14:50:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.259752106s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..XXXX......................................................] duration: 8.364700011s FAIL (2025-09-24 14:49:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 8.364735458s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...XXXX.....................................................] duration: 14.409257715s FAIL (2025-09-24 14:48:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 14.409362274s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....XXXX....................................................] duration: 8.815687456s FAIL (2025-09-24 14:47:50 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 8.815747579s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....XXXX...................................................] duration: 14.377950138s FAIL (2025-09-24 14:46:41 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 14.377999402s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......XXXX..................................................] duration: 10.14333654s FAIL (2025-09-24 14:45:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 10.14336802s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......XXXX.................................................] duration: 7.03394451s FAIL (2025-09-24 14:45:19 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.03398185s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........XXXX................................................] duration: 9.911321905s FAIL (2025-09-24 14:44:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 9.911354578s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........XXXX...............................................] duration: 7.183125155s FAIL (2025-09-24 14:43:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.183160272s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........XXXX..............................................] duration: 12.372817964s FAIL (2025-09-24 14:43:21 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 12.372850174s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........XXXX.............................................] duration: 7.740560813s FAIL (2025-09-24 14:42:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.740591313s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............XXXX............................................] duration: 6.451726478s FAIL (2025-09-24 14:42:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.451762848s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............XXXX...........................................] duration: 6.490133606s FAIL (2025-09-24 14:41:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.490229939s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............XXXX..........................................] duration: 10.171085122s FAIL (2025-09-24 14:40:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 10.171120327s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............XXXX.........................................] duration: 6.916142061s FAIL (2025-09-24 14:39:43 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.916179633s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................XXXX........................................] duration: 7.202261963s FAIL (2025-09-24 14:39:10 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.20229709s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................XXXX.......................................] duration: 7.51106751s FAIL (2025-09-24 14:38:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.511111303s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................XXXX......................................] duration: 7.46701159s FAIL (2025-09-24 14:38:17 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.467157688s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................XXXX.....................................] duration: 6.488766457s FAIL (2025-09-24 14:37:20 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.488799801s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................XXXX....................................] duration: 12.035489918s FAIL (2025-09-24 14:36:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 12.035523952s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................XXXX...................................] duration: 7.465906667s FAIL (2025-09-24 14:36:24 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.465938077s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................XXXX..................................] duration: 12.048510707s FAIL (2025-09-24 14:35:35 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 12.048541516s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................XXXX.................................] duration: 6.742231008s FAIL (2025-09-24 14:34:31 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.742260896s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................XXXX................................] duration: 12.344568312s FAIL (2025-09-24 14:34:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 12.34459369s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.........................XXXX...............................] duration: 8.352187429s FAIL (2025-09-24 14:32:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 8.352223237s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..........................XXXX..............................] duration: 6.53564231s FAIL (2025-09-24 14:32:00 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.535676002s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...........................XXXX.............................] duration: 11.799731131s FAIL (2025-09-24 14:31:18 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 11.799765617s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [............................XXXX............................] duration: 7.082468747s FAIL (2025-09-24 14:30:54 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.082506699s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................XXXX...........................] duration: 7.842640186s FAIL (2025-09-24 14:30:07 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.842671796s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................XXXX..........................] duration: 8.519005559s FAIL (2025-09-24 14:29:26 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 8.519039644s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................XXXX.........................] duration: 6.9065349s FAIL (2025-09-24 14:28:55 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.906567233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................XXXX........................] duration: 7.381022433s FAIL (2025-09-24 14:27:57 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.381058591s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................XXXX.......................] duration: 7.018619026s FAIL (2025-09-24 14:26:49 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.018648052s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................XXXX......................] duration: 7.811304942s FAIL (2025-09-24 14:26:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.811335219s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................XXXX.....................] duration: 6.909083031s FAIL (2025-09-24 14:25:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.909114981s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................XXXX....................] duration: 6.233356922s FAIL (2025-09-24 14:25:13 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.233394663s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................XXXX...................] duration: 10.966746993s FAIL (2025-09-24 14:24:46 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 10.966779784s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................XXXX..................] duration: 6.478016769s FAIL (2025-09-24 14:23:40 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.478082773s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................XXXX.................] duration: 6.037653834s FAIL (2025-09-24 14:23:22 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.037689421s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)