Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.............................................XXXX...........] duration: 7.135375795s FAIL (2025-09-24 14:18:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 7.13540974s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..............................................XXXX..........] duration: 6.169895179s FAIL (2025-09-24 14:17:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.169985331s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...............................................XXXX.........] duration: 6.139007828s FAIL (2025-09-24 14:16:53 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.139045279s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [................................................XXXX........] duration: 6.057217904s FAIL (2025-09-24 14:16:33 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.057249775s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.................................................XXXX.......] duration: 8.289320674s FAIL (2025-09-24 14:15:27 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 8.289349198s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [..................................................XXXX......] duration: 11.538435438s FAIL (2025-09-24 14:15:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 11.538467849s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [...................................................XXXX.....] duration: 6.197452504s FAIL (2025-09-24 14:13:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.197487561s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [....................................................XXXX....] duration: 11.888323978s FAIL (2025-09-24 14:13:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 11.888357061s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.....................................................XXXX...] duration: 11.585587964s FAIL (2025-09-24 14:12:04 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 11.585628361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [......................................................XXXX..] duration: 11.273032594s FAIL (2025-09-24 14:11:03 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 11.273069303s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.......................................................XXXX.] duration: 6.808393445s FAIL (2025-09-24 14:10:05 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 6.80842731s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [........................................................XXXX] duration: 30.001784101s FAIL (2025-09-24 14:09:23 UTC) orbiter warning: operation: ListObjects ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== 1682]: Done starting upload Sep 24 14:08:54.106670160 (24h37m39.568762643s) [upload=21682]: Uploading segments... Sep 24 14:08:54.106781952 (24h37m39.568874586s) [upload=21682]: Got next segment. Inline: false Sep 24 14:08:54.106814114 (24h37m39.568906597s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:08:54.106820025 (24h37m39.568912508s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:08:54.368052647 (24h37m39.83014513s) [seg_pos={0 0}, upload=21682]: Begin upload segment... Sep 24 14:08:54.368163158 (24h37m39.830255641s) [seg_pos={0 0}, upload=21682]: Done begin upload segment. Sep 24 14:08:54.368194207 (24h37m39.83028669s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368238161 (24h37m39.830330634s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.368785082 (24h37m39.830877555s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.369166699 (24h37m39.831259182s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370101859 (24h37m39.832194333s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.370227098 (24h37m39.832319581s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.371308858 (24h37m39.833401331s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.372969449 (24h37m39.835061923s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.374959600 (24h37m39.837052093s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.376274262 (24h37m39.838366746s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378388318 (24h37m39.840480801s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.378522103 (24h37m39.840614586s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.380180101 (24h37m39.842272584s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381447353 (24h37m39.843539836s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.381507378 (24h37m39.843599861s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:54.382023771 (24h37m39.844116244s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Uploading piece... Sep 24 14:08:56.215258635 (24h37m41.677351108s) [upload=21682]: Next returned nil segment Sep 24 14:08:56.215278153 (24h37m41.677370636s) [upload=21682]: Waiting for error group managing segments... Sep 24 14:08:56.501403055 (24h37m41.963495539s) [noise=true , address=77.90.31.10:4518 , piece_id=IVPFRSLUKJR67WBYU2PQYIRC6WWNCIRG7EO52NYZCYC2QFAZ7VDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.519105303 (24h37m41.981197776s) [noise=true , address=45.91.116.5:4299 , piece_id=UDBIWEWO7D6MTCLNBI565SRCBSC5PSX52JIVYXHINYKOZZ5HC4DQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.557166659 (24h37m42.019259132s) [noise=true , address=77.90.31.11:4554 , piece_id=RSUMR7Y347YTB6LFLOGMZSBIKQL674Y7DX2X46UBAFHVKHTTT5JA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.563643332 (24h37m42.025735805s) [noise=true , address=45.91.116.4:4938 , piece_id=EXN7OXZ7I3DLCQGSALYBC74NPNKA5KMYITOLPZ4OXUOY4EPAKOXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.565778858 (24h37m42.027871341s) [noise=true , address=45.91.116.8:4045 , piece_id=RY3D7NIMNO2A6RDEQ7Z5QFMKNXDQWWQZSI6OJ4D3DR7BC3EW3T3A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.580729417 (24h37m42.04282189s) [noise=true , address=77.90.31.16:4734 , piece_id=UB6TSWBJZQC6AVEKQYHJGCHP3SE6BN7HH4SOL3LARX5XOU5UMK5A, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.610388242 (24h37m42.072480725s) [noise=true , address=77.90.31.18:4807 , piece_id=SW65ISPX6F3ME5PU3PGPIOREI4H57CTAOOF37VVH5BLCB3KSXIFA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.613857599 (24h37m42.075950072s) [noise=true , address=77.90.31.19:4842 , piece_id=XCORWUJLHC3FQF4MA3IDJKRWH4RULVDXVBM7773Y2K67FC6EF7LA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.645583400 (24h37m42.107675883s) [noise=true , address=45.91.116.7:4010 , piece_id=5A27EOYNPL76BYUDGE46SBDQ457S3EWKYN5KXIVPHO3FKBEFOCGQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.657837193 (24h37m42.119929676s) [noise=true , address=77.90.31.11:4555 , piece_id=VKYFV4YKAOIZZVM3IJE3V3AFNVLKN3YPCJBQQ3P4XPP635Q4LZ2Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:56.668562296 (24h37m42.130654779s) [noise=true , address=77.90.31.21:4877 , piece_id=CTPTGPVOOTN55HTNYJUX74DICPRQUD36UFJA5R564H7TH6W2JUXQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:08:57.370380272 (24h37m42.832472755s) [noise=true , address=45.91.116.6:4975 , piece_id=VIISNW7FZCEYJKR6MB6ZEHHGKFHTVDUTMIQGNU43OHG4EUJSDS4Q, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823565560 (24h38m8.285658043s) [noise=true , address=145.79.227.15:4875 , piece_id=3RPOSD24VXKOXKV7P2QJ6S7HOLMEVFJQAIREMDKTBE4IUG5BPWIQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: <nil> Sep 24 14:09:22.823605125 (24h38m8.285697598s) [seg_pos={0 0}, upload=21682]: Segment reached optimal threshold of 13 pieces. Sep 24 14:09:22.823769679 (24h38m8.285862172s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12qexRLJM6TY935UaEWsj1X9ViyBvk2BM1XGjnq8kapnXXXmgbm): protocol: expected piece hash; context canceled; EOF Sep 24 14:09:22.823780058 (24h38m8.285872541s) [noise=true , address=145.79.227.3:4149 , piece_id=NQQDFG2ACKRDWCRRIDWJO3BS5AREPCIFBUVKFI34TPWMTCR2ZIDA, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823798013 (24h38m8.285890516s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1ET8PaAfWq2RcPNjucMqDbmnkMZg8qnkTw9dszwLzz765oBr7y): protocol: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52970->145.79.227.6:4259: use of closed network connection Sep 24 14:09:22.823809605 (24h38m8.285902078s) [noise=true , address=145.79.227.6:4259 , piece_id=LGDKOJ5C4V37E3DSZ5B7IYS3XM2YERAWFZ4CRBPMLOUJ752W7LRQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823884668 (24h38m8.285977151s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu): protocol: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48800->145.79.227.16:4911: use of closed network connection Sep 24 14:09:22.823892473 (24h38m8.285984956s) [noise=true , address=145.79.227.16:4911 , piece_id=KPPZBBXSGK745KBH6SNFOWLE3DTLL52UDOFQE4BNUI4VUCZ2PCMQ, seg_pos={0 0}, upload=21682]: Long tail cancellation due to optimal threshold Sep 24 14:09:22.823907652 (24h38m8.286000135s) [seg_pos={0 0}, upload=21682]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Sep 24 14:09:22.823986912 (24h38m8.286079395s) [upload=21682]: Flush batch item: pb.BatchRequestItem_SegmentCommit Sep 24 14:09:22.823992563 (24h38m8.286085046s) [upload=21682]: Flush batch item: pb.BatchRequestItem_ObjectCommit Sep 24 14:09:23.150780827 (24h38m8.6128733s) [upload=21682]: Done uploading segments... Sep 24 14:09:23.150811355 (24h38m8.612903828s) [upload=21682]: Upload finished. err: <nil> Duration: 30.001807304s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [.........................................................XXX] duration: 30.271626321s FAIL (2025-09-24 14:08:11 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 11) ======LOGS====== Sep 24 14:07:41.780605590 (24h36m27.242698073s) [upload=21668]: Starting upload Sep 24 14:07:41.780740157 (24h36m27.24283264s) [upload=21668]: Done starting upload Sep 24 14:07:41.780853542 (24h36m27.242946015s) [upload=21668]: Uploading segments... Sep 24 14:07:41.781018797 (24h36m27.24311128s) [upload=21668]: Got next segment. Inline: false Sep 24 14:07:41.781080605 (24h36m27.243173088s) [upload=21668]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:07:41.781088039 (24h36m27.243180522s) [upload=21668]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:07:42.042314261 (24h36m27.504406744s) [seg_pos={0 0}, upload=21668]: Begin upload segment... Sep 24 14:07:42.042477060 (24h36m27.504569543s) [seg_pos={0 0}, upload=21668]: Done begin upload segment. Sep 24 14:07:42.042490196 (24h36m27.504582679s) [noise=true , address=45.91.116.7:4010 , piece_id=HWON352WIP6KKV4SVPDWOSMU5ZNASCK5ZN6MN2LNWAIBUUHDPVLQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.042507639 (24h36m27.504600122s) [noise=true , address=145.79.227.11:4729 , piece_id=AFPVCVSIUYXUOCXDD7FMEWDTRVHPTJY3HAHZOR5DAPETVY4H4TCQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.042586218 (24h36m27.504678701s) [noise=true , address=45.91.116.6:4973 , piece_id=VY5DTOBXHDDLHLQGXVHMFE47GTLB6GCJQHRCLCOT347S7G5E2F6A, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.043060201 (24h36m27.505152684s) [noise=true , address=45.91.116.5:4299 , piece_id=2XLV4ETPMMMW4B46RS6LEB4N5HJVO6I7SXDQZFJC3NQKIQNTBZ3Q, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.043442579 (24h36m27.505535062s) [noise=true , address=77.90.31.10:4517 , piece_id=G5VS6GYZCMDPPIO4TGV5Z7CYRNDEFVLG5PKC6S2T3KLD72HFLPIA, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.043840728 (24h36m27.505933221s) [noise=true , address=145.79.227.16:4911 , piece_id=ZHHEZUXW6BDJFBJGETYBQ4H3SKRWJ4PNXTJ3K5K62LCMJ7Q65K3A, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.044358413 (24h36m27.506450896s) [noise=true , address=145.79.227.11:4732 , piece_id=6L7BE3C5J3OQ3563EWR2PODQYCN7J35DOLDON3EYMFEAB2KPWAZQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.044802258 (24h36m27.506894741s) [noise=true , address=45.91.116.4:4937 , piece_id=6BNUKDTFU23QHT2BCLAAXTXC4UKQIOFH6SRGIAS2TTE72KKUBKEQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.045392682 (24h36m27.507485165s) [noise=true , address=45.91.116.14:4191 , piece_id=7HKCFT6U34DKSTFZUZYQOBSWIKHYPBPKTO54SEHSLQGNIYY3MKKQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.045948741 (24h36m27.508041224s) [noise=true , address=145.79.227.15:4874 , piece_id=KCU3W4GTK2MZQ7SGZUPLWY4EGVIEA7NJ4UDGHHS5XLSXCTM44UWQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.047260960 (24h36m27.509353433s) [noise=true , address=45.91.116.10:4081 , piece_id=GWZGF5D4UQUGGUE6QC2YOLUFFITUCI2DXTT6VK7RXQ2KRCC3I6FA, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.047757475 (24h36m27.509849968s) [noise=true , address=45.91.116.11:4117 , piece_id=5AIUISS436G2JE2HZJZSQ2IRZ7PT5KKBC5P6T6DKC22LQPRWV2IQ, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.048850256 (24h36m27.510942739s) [noise=true , address=77.90.31.19:4842 , piece_id=OK6NEEVS75TYG5DAMRQJJMLH7GXFO3SN3DQ7DZ3PBKGJFAIXPFUA, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.049500685 (24h36m27.511593168s) [noise=true , address=145.79.227.16:4910 , piece_id=WOVNS27D2DZEE23BIWQWOIHDRC57ZCFXSHVKLS46X6AXETDCETBA, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.050535596 (24h36m27.512628079s) [noise=true , address=45.91.116.12:4226 , piece_id=JISGR653YENKR5KW6ZYTTKEMKNFD5VNAYPIM7K4VQVHI7LRPS45A, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:42.052389996 (24h36m27.514482479s) [noise=true , address=77.90.31.17:4771 , piece_id=BX3RIDMGBUEA5VSLC6KVU5REDGZ5QCE3QPUUUBFCS22NNVF5VZZA, seg_pos={0 0}, upload=21668]: Uploading piece... Sep 24 14:07:43.896638094 (24h36m29.358730567s) [upload=21668]: Next returned nil segment Sep 24 14:07:43.896662470 (24h36m29.358754953s) [upload=21668]: Waiting for error group managing segments... Sep 24 14:07:44.189299894 (24h36m29.651392367s) [noise=true , address=45.91.116.14:4191 , piece_id=7HKCFT6U34DKSTFZUZYQOBSWIKHYPBPKTO54SEHSLQGNIYY3MKKQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.241697428 (24h36m29.703789901s) [noise=true , address=45.91.116.5:4299 , piece_id=2XLV4ETPMMMW4B46RS6LEB4N5HJVO6I7SXDQZFJC3NQKIQNTBZ3Q, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.254690909 (24h36m29.716783393s) [noise=true , address=45.91.116.12:4226 , piece_id=JISGR653YENKR5KW6ZYTTKEMKNFD5VNAYPIM7K4VQVHI7LRPS45A, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.277993362 (24h36m29.740085845s) [noise=true , address=77.90.31.10:4517 , piece_id=G5VS6GYZCMDPPIO4TGV5Z7CYRNDEFVLG5PKC6S2T3KLD72HFLPIA, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.282099701 (24h36m29.744192184s) [noise=true , address=45.91.116.6:4973 , piece_id=VY5DTOBXHDDLHLQGXVHMFE47GTLB6GCJQHRCLCOT347S7G5E2F6A, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.290860043 (24h36m29.752952526s) [noise=true , address=45.91.116.11:4117 , piece_id=5AIUISS436G2JE2HZJZSQ2IRZ7PT5KKBC5P6T6DKC22LQPRWV2IQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.296744558 (24h36m29.758837031s) [noise=true , address=45.91.116.10:4081 , piece_id=GWZGF5D4UQUGGUE6QC2YOLUFFITUCI2DXTT6VK7RXQ2KRCC3I6FA, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.306470748 (24h36m29.768563221s) [noise=true , address=77.90.31.19:4842 , piece_id=OK6NEEVS75TYG5DAMRQJJMLH7GXFO3SN3DQ7DZ3PBKGJFAIXPFUA, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.311736955 (24h36m29.773829428s) [noise=true , address=45.91.116.4:4937 , piece_id=6BNUKDTFU23QHT2BCLAAXTXC4UKQIOFH6SRGIAS2TTE72KKUBKEQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.319789630 (24h36m29.781882103s) [noise=true , address=45.91.116.7:4010 , piece_id=HWON352WIP6KKV4SVPDWOSMU5ZNASCK5ZN6MN2LNWAIBUUHDPVLQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:07:44.327746611 (24h36m29.789839085s) [noise=true , address=77.90.31.17:4771 , piece_id=BX3RIDMGBUEA5VSLC6KVU5REDGZ5QCE3QPUUUBFCS22NNVF5VZZA, seg_pos={0 0}, upload=21668]: Done uploading piece. err: <nil> Sep 24 14:08:11.097286817 (24h36m56.55937931s) [noise=true , address=145.79.227.16:4911 , piece_id=ZHHEZUXW6BDJFBJGETYBQ4H3SKRWJ4PNXTJ3K5K62LCMJ7Q65K3A, seg_pos={0 0}, upload=21668]: Done uploading piece. err: ecclient: upload failed (node:13rWy7bK7AMZnejVCPwyMs7QrbNcSjWpuVEL8dKX2awG4oJUqu, address:145.79.227.16:4911): protocol: write tcp 188.166.225.170:35608->145.79.227.16:4911: use of closed network connection; write tcp 188.166.225.170:35608->145.79.227.16:4911: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:35608->145.79.227.16:4911: use of closed network connection Sep 24 14:08:11.097414821 (24h36m56.559507314s) [noise=true , address=145.79.227.15:4874 , piece_id=KCU3W4GTK2MZQ7SGZUPLWY4EGVIEA7NJ4UDGHHS5XLSXCTM44UWQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: expected piece hash; context deadline exceeded; EOF Sep 24 14:08:11.097456811 (24h36m56.559549294s) [noise=true , address=145.79.227.16:4910 , piece_id=WOVNS27D2DZEE23BIWQWOIHDRC57ZCFXSHVKLS46X6AXETDCETBA, seg_pos={0 0}, upload=21668]: Done uploading piece. err: ecclient: upload failed (node:15uiNvgWc1hPsws7NJjiH5dTx8aBh6PG4fQhsNQTqQTi9MC2Hn, address:145.79.227.16:4910): protocol: write tcp 188.166.225.170:47522->145.79.227.16:4910: use of closed network connection; write tcp 188.166.225.170:47522->145.79.227.16:4910: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47522->145.79.227.16:4910: use of closed network connection Sep 24 14:08:11.097527625 (24h36m56.559620108s) [noise=true , address=145.79.227.11:4732 , piece_id=6L7BE3C5J3OQ3563EWR2PODQYCN7J35DOLDON3EYMFEAB2KPWAZQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: ecclient: upload failed (node:12KuoS4Fh3YKpoEgZuxcDLE1e1KZENgrLTqR9T5yCWonHiVfaXE, address:145.79.227.11:4732): protocol: write tcp 188.166.225.170:46038->145.79.227.11:4732: use of closed network connection; write tcp 188.166.225.170:46038->145.79.227.11:4732: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:46038->145.79.227.11:4732: use of closed network connection Sep 24 14:08:11.097570958 (24h36m56.559663441s) [noise=true , address=145.79.227.11:4729 , piece_id=AFPVCVSIUYXUOCXDD7FMEWDTRVHPTJY3HAHZOR5DAPETVY4H4TCQ, seg_pos={0 0}, upload=21668]: Done uploading piece. err: ecclient: upload failed (node:1BKzosFpHWqYNaAck7XPxK263dJ4bPZ6hVSYye65CJa1uHRmoR, address:145.79.227.11:4729): protocol: write tcp 188.166.225.170:52072->145.79.227.11:4729: use of closed network connection; write tcp 188.166.225.170:52072->145.79.227.11:4729: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:52072->145.79.227.11:4729: use of closed network connection Sep 24 14:08:11.097615132 (24h36m56.559707615s) [seg_pos={0 0}, upload=21668]: Done waiting for segment. successful: 11 optimal: 13 errs: <nil> Sep 24 14:08:11.367350355 (24h36m56.829442838s) [upload=21668]: Done uploading segments... Sep 24 14:08:11.367390562 (24h36m56.829483045s) [upload=21668]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 11) Duration: 30.271728036s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.238372241s FAIL (2025-09-24 14:07:12 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 24 14:06:42.623257355 (24h35m28.085349838s) [upload=21653]: Starting upload Sep 24 14:06:42.623414364 (24h35m28.085506857s) [upload=21653]: Done starting upload Sep 24 14:06:42.623561684 (24h35m28.085654158s) [upload=21653]: Uploading segments... Sep 24 14:06:42.623682595 (24h35m28.085775078s) [upload=21653]: Got next segment. Inline: false Sep 24 14:06:42.623775812 (24h35m28.085868315s) [upload=21653]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:06:42.623784610 (24h35m28.085877083s) [upload=21653]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:06:42.869875149 (24h35m28.331967642s) [seg_pos={0 0}, upload=21653]: Begin upload segment... Sep 24 14:06:42.869968086 (24h35m28.332060559s) [seg_pos={0 0}, upload=21653]: Done begin upload segment. Sep 24 14:06:42.869985610 (24h35m28.332078093s) [noise=true , address=77.90.31.10:4518 , piece_id=JRMEO2OFJJSHBTU5P2R2ZPGDDEPTN42KK5NS6KN45CX5HLY7YSWQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.869999336 (24h35m28.332091819s) [noise=true , address=45.91.116.5:4297 , piece_id=TWKALFHSZ2OYBXAPZ2X24XEKBOVL2PFS7HJLNPT5G7DV6BD3UWMQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.870689891 (24h35m28.332782374s) [noise=true , address=45.91.116.13:4155 , piece_id=EN2LMOGSVY35JC3WDDNB5TBNOJMDAZLJBKVGYOUHFG73E26JR4XQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.871158904 (24h35m28.333251397s) [noise=true , address=45.91.116.4:4938 , piece_id=GUSDU573QA42FZR7TIZUFOOAK63T5WLFKAJLKBGOTRWP65HUOFSA, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.871713610 (24h35m28.333806093s) [noise=true , address=145.79.227.5:4221 , piece_id=OMPNS7JBRRWNZQ6UZIIEMJGDUX2CE5DWVH2UIQXHWX3POXCYQ2NQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.872938011 (24h35m28.335030494s) [noise=true , address=45.91.116.8:4047 , piece_id=EQYRG5B3QNQBWN2IKALST2BHUTIOMFUP3RKGJ6HXNQAHJMBUIF7A, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.873497206 (24h35m28.335589689s) [noise=true , address=145.79.227.19:4079 , piece_id=5K6IQ5AS6JKLN5YOVXBTOPTTKU5U6FXRD26Q7WTPGVUPDNYBQOAQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.874092270 (24h35m28.336184753s) [noise=true , address=145.79.227.4:4185 , piece_id=MTUQS545R2MEQF45YCIJXQSWLOVC4OGQYU7I6ZZ5MN3JA3QX6MOQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.874670751 (24h35m28.336763234s) [noise=true , address=77.90.31.14:4662 , piece_id=OIKCHUYJ363RSFLHOWSKYPCSNT3CIFPL4X2BNSZIRPUHPNKLLJWQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.877107491 (24h35m28.339199974s) [noise=true , address=45.91.116.11:4119 , piece_id=DRM6OLRUIKHLXU3DR7MRCLYJKFUDMMYFDW2D6O6AKITM5TCI2GUQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.880099629 (24h35m28.342192112s) [noise=true , address=77.90.31.13:4627 , piece_id=TISUCXOXC6ZOHUFZPPWAG24IG7ZUHF4SRST5TRL3YZWD5B3LSKOQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.882708707 (24h35m28.34480119s) [noise=true , address=45.91.116.12:4225 , piece_id=FFLADU7VDL7WQRS6FNFOXWHXSIISKR3EJN2QO6JECAAQ2A7AMWGQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.884839394 (24h35m28.346931877s) [noise=true , address=77.90.31.14:4661 , piece_id=TGANDMYOZSIVQGSXTJPPMPCYSQTAAOQTTJR5UA5GAABHYHZDRNHQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.886738029 (24h35m28.348830523s) [noise=true , address=77.90.31.18:4807 , piece_id=7MSIAOTSMIOZQNWWPEFVR4LFOD52QAOOGVJ7KWLGYXJ55Y2UWYIQ, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.888747035 (24h35m28.350839518s) [noise=true , address=145.79.227.4:4186 , piece_id=UDDMVUMYRSMI7C3BZ3DIMEFI6SP6IQZ2GYWYVIOPRNKIGMQJMH4A, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:42.892018114 (24h35m28.354110597s) [noise=true , address=77.90.31.17:4772 , piece_id=YHTFM5ZLIK6MZ5EW246TU442PQVEL7FQOBTVRKA7Q7XSWDZM4F4A, seg_pos={0 0}, upload=21653]: Uploading piece... Sep 24 14:06:44.752663143 (24h35m30.214755626s) [upload=21653]: Next returned nil segment Sep 24 14:06:44.752712446 (24h35m30.21480493s) [upload=21653]: Waiting for error group managing segments... Sep 24 14:06:45.042627867 (24h35m30.50472035s) [noise=true , address=77.90.31.14:4662 , piece_id=OIKCHUYJ363RSFLHOWSKYPCSNT3CIFPL4X2BNSZIRPUHPNKLLJWQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.063803229 (24h35m30.525895712s) [noise=true , address=77.90.31.17:4772 , piece_id=YHTFM5ZLIK6MZ5EW246TU442PQVEL7FQOBTVRKA7Q7XSWDZM4F4A, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.068876901 (24h35m30.530969384s) [noise=true , address=45.91.116.13:4155 , piece_id=EN2LMOGSVY35JC3WDDNB5TBNOJMDAZLJBKVGYOUHFG73E26JR4XQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.073701627 (24h35m30.53579411s) [noise=true , address=77.90.31.10:4518 , piece_id=JRMEO2OFJJSHBTU5P2R2ZPGDDEPTN42KK5NS6KN45CX5HLY7YSWQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.105513433 (24h35m30.567605906s) [noise=true , address=45.91.116.4:4938 , piece_id=GUSDU573QA42FZR7TIZUFOOAK63T5WLFKAJLKBGOTRWP65HUOFSA, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.114469737 (24h35m30.57656222s) [noise=true , address=45.91.116.12:4225 , piece_id=FFLADU7VDL7WQRS6FNFOXWHXSIISKR3EJN2QO6JECAAQ2A7AMWGQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.121752906 (24h35m30.583845379s) [noise=true , address=45.91.116.8:4047 , piece_id=EQYRG5B3QNQBWN2IKALST2BHUTIOMFUP3RKGJ6HXNQAHJMBUIF7A, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.139141526 (24h35m30.601233999s) [noise=true , address=45.91.116.5:4297 , piece_id=TWKALFHSZ2OYBXAPZ2X24XEKBOVL2PFS7HJLNPT5G7DV6BD3UWMQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.140983584 (24h35m30.603076067s) [noise=true , address=77.90.31.18:4807 , piece_id=7MSIAOTSMIOZQNWWPEFVR4LFOD52QAOOGVJ7KWLGYXJ55Y2UWYIQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.164708401 (24h35m30.626800884s) [noise=true , address=45.91.116.11:4119 , piece_id=DRM6OLRUIKHLXU3DR7MRCLYJKFUDMMYFDW2D6O6AKITM5TCI2GUQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.185088108 (24h35m30.647180591s) [noise=true , address=77.90.31.14:4661 , piece_id=TGANDMYOZSIVQGSXTJPPMPCYSQTAAOQTTJR5UA5GAABHYHZDRNHQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:06:45.237624616 (24h35m30.699717089s) [noise=true , address=77.90.31.13:4627 , piece_id=TISUCXOXC6ZOHUFZPPWAG24IG7ZUHF4SRST5TRL3YZWD5B3LSKOQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: <nil> Sep 24 14:07:11.940367347 (24h35m57.40245984s) [noise=true , address=145.79.227.5:4221 , piece_id=OMPNS7JBRRWNZQ6UZIIEMJGDUX2CE5DWVH2UIQXHWX3POXCYQ2NQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: ecclient: upload failed (node:129qMN1kX8C3x3H1Epfc4pUbtVGU1xPiZj43QSLByCTPKqwUoJR, address:145.79.227.5:4221): protocol: write tcp 188.166.225.170:59462->145.79.227.5:4221: use of closed network connection; write tcp 188.166.225.170:59462->145.79.227.5:4221: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:59462->145.79.227.5:4221: use of closed network connection Sep 24 14:07:11.940410019 (24h35m57.402502492s) [noise=true , address=145.79.227.4:4185 , piece_id=MTUQS545R2MEQF45YCIJXQSWLOVC4OGQYU7I6ZZ5MN3JA3QX6MOQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: ecclient: upload failed (node:1J4E8bxy3XRU8MmG8Jn3g5DiiBAGs6smUfYSgRea3mbCTZVgDN, address:145.79.227.4:4185): protocol: expected piece hash; context deadline exceeded; EOF Sep 24 14:07:11.940480443 (24h35m57.402572916s) [noise=true , address=145.79.227.4:4186 , piece_id=UDDMVUMYRSMI7C3BZ3DIMEFI6SP6IQZ2GYWYVIOPRNKIGMQJMH4A, seg_pos={0 0}, upload=21653]: Done uploading piece. err: ecclient: upload failed (node:16iUE9RyvrbkRAv7DZVHPsf9fk5ukduiYriLkcS43YNgxu66ht, address:145.79.227.4:4186): protocol: write tcp 188.166.225.170:60162->145.79.227.4:4186: use of closed network connection; write tcp 188.166.225.170:60162->145.79.227.4:4186: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60162->145.79.227.4:4186: use of closed network connection Sep 24 14:07:11.940504498 (24h35m57.402596982s) [noise=true , address=145.79.227.19:4079 , piece_id=5K6IQ5AS6JKLN5YOVXBTOPTTKU5U6FXRD26Q7WTPGVUPDNYBQOAQ, seg_pos={0 0}, upload=21653]: Done uploading piece. err: ecclient: upload failed (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud, address:145.79.227.19:4079): protocol: write tcp 188.166.225.170:48046->145.79.227.19:4079: use of closed network connection; write tcp 188.166.225.170:48046->145.79.227.19:4079: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:48046->145.79.227.19:4079: use of closed network connection Sep 24 14:07:11.940530859 (24h35m57.402623342s) [seg_pos={0 0}, upload=21653]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 24 14:07:12.176977737 (24h35m57.63907021s) [upload=21653]: Done uploading segments... Sep 24 14:07:12.177007904 (24h35m57.639100377s) [upload=21653]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.238389364s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.241296493s FAIL (2025-09-24 14:06:41 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 12) ======LOGS====== Sep 24 14:06:12.166086577 (24h34m57.62817906s) [upload=21642]: Starting upload Sep 24 14:06:12.166154186 (24h34m57.628246669s) [upload=21642]: Done starting upload Sep 24 14:06:12.166240340 (24h34m57.628332833s) [upload=21642]: Uploading segments... Sep 24 14:06:12.166460199 (24h34m57.628552702s) [upload=21642]: Got next segment. Inline: false Sep 24 14:06:12.166502479 (24h34m57.628594962s) [upload=21642]: Flush batch item: pb.BatchRequestItem_ObjectBegin Sep 24 14:06:12.166511586 (24h34m57.628604069s) [upload=21642]: Flush batch item: pb.BatchRequestItem_SegmentBegin Sep 24 14:06:12.437943060 (24h34m57.900035543s) [seg_pos={0 0}, upload=21642]: Begin upload segment... Sep 24 14:06:12.438010378 (24h34m57.900102851s) [seg_pos={0 0}, upload=21642]: Done begin upload segment. Sep 24 14:06:12.438040486 (24h34m57.900132969s) [noise=true , address=45.91.116.9:4262 , piece_id=IQAHEVE2REPXG3YRUBCSNRNFMOGRZSEW36GQF4BRBS6SLNKLVXUA, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.438144514 (24h34m57.900236987s) [noise=true , address=45.91.116.8:4047 , piece_id=4VEYC72WDBDFYFCXD5C5YSCYFNLCCG65F54ENBAOCJHDNNGUOYKQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.439024971 (24h34m57.901117464s) [noise=true , address=145.79.227.15:4874 , piece_id=XHH6XEHPS5UAKYV4GBOGSW5H4PNPRRTRK23UAZVQ5JBJQMNVVVHA, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.439035511 (24h34m57.901127994s) [noise=true , address=45.91.116.8:4048 , piece_id=ZQJFCSNZFZV44IWFFZ6L6EEAWU2PE3JPTG5UNU6AYXZHMXL6PQSQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.439441214 (24h34m57.901533697s) [noise=true , address=77.90.31.11:4554 , piece_id=L6RJOID3MXSYA2IZLSNJR3D5THMR7WEAC32HHHNUQI5HMMSRX6TQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.439620395 (24h34m57.901712878s) [noise=true , address=45.91.116.6:4973 , piece_id=RPGLDUJH5RVGTA57XUAUT7VDZHHXANEWDT6BHXUPE6SE3RF3FGSA, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.439865271 (24h34m57.901957744s) [noise=true , address=45.91.116.5:4299 , piece_id=3RBI4K5PYX6SHMMGWKX4NGOKVGL2MMSSZK3OOPO7O6ASTEKMVCNA, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.440408545 (24h34m57.902501028s) [noise=true , address=77.90.31.15:4698 , piece_id=3Z5SNODVDVEBDNLJIBQY6W2UUL6C253EFLOHI4QQSBQG3REHLANQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.440425578 (24h34m57.902518061s) [noise=true , address=145.79.227.5:4223 , piece_id=YULOT23Y5JT3YRRGKXKPIGRBH4ZCATDQDNZZMYSTZ64XNJ7RVNYQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.440830428 (24h34m57.902922911s) [noise=true , address=145.79.227.6:4258 , piece_id=7DH7B3UOTJ5LU5PSEDA23P7NZBVMCQ4MMI7NSRP4VA2VD2L32C4Q, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.441213798 (24h34m57.903306281s) [noise=true , address=77.90.31.10:4518 , piece_id=O2IDUCHQLU6PAY2YE25HQ477QWECQXFFYHRCOTKX2X5NXNKE326Q, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.441234568 (24h34m57.903327051s) [noise=true , address=145.79.227.13:4801 , piece_id=KNS7UF4X4WB55YP7EZAR6XLU2C4WKIMUTFMPFGRTBO7IJNOEJ46A, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.441652474 (24h34m57.903744957s) [noise=true , address=45.91.116.12:4226 , piece_id=D54EQ4HTI63POCYN7A5UKPX2LGKFNWVM4D6ETVQFHOTTMFPFJ7YQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.442068075 (24h34m57.904160558s) [noise=true , address=45.91.116.11:4119 , piece_id=SSMVXKOZJVM25DEKWO5LGID6PXM5MC72YNEHDOSTWBDXRPJOIJQQ, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.442853030 (24h34m57.904945514s) [noise=true , address=77.90.31.17:4770 , piece_id=6UAU4NX2FGUZUR4UYCANTMFJWJPVXSTHR24DLUI7EHJTZBQOYWYA, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:12.443470546 (24h34m57.905563019s) [noise=true , address=77.90.31.10:4517 , piece_id=RX56BTEMG7IJI3BZISZ5I3KIPYE4A5CHGNKG22LMVVBWHMNA7O6A, seg_pos={0 0}, upload=21642]: Uploading piece... Sep 24 14:06:14.291282499 (24h34m59.753375002s) [upload=21642]: Next returned nil segment Sep 24 14:06:14.291305092 (24h34m59.753397575s) [upload=21642]: Waiting for error group managing segments... Sep 24 14:06:14.586093311 (24h35m0.048185794s) [noise=true , address=45.91.116.8:4048 , piece_id=ZQJFCSNZFZV44IWFFZ6L6EEAWU2PE3JPTG5UNU6AYXZHMXL6PQSQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.614004039 (24h35m0.076096522s) [noise=true , address=77.90.31.11:4554 , piece_id=L6RJOID3MXSYA2IZLSNJR3D5THMR7WEAC32HHHNUQI5HMMSRX6TQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.667380837 (24h35m0.12947332s) [noise=true , address=45.91.116.12:4226 , piece_id=D54EQ4HTI63POCYN7A5UKPX2LGKFNWVM4D6ETVQFHOTTMFPFJ7YQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.676240808 (24h35m0.138333291s) [noise=true , address=77.90.31.17:4770 , piece_id=6UAU4NX2FGUZUR4UYCANTMFJWJPVXSTHR24DLUI7EHJTZBQOYWYA, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.718905178 (24h35m0.180997661s) [noise=true , address=77.90.31.10:4517 , piece_id=RX56BTEMG7IJI3BZISZ5I3KIPYE4A5CHGNKG22LMVVBWHMNA7O6A, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.746068713 (24h35m0.208161196s) [noise=true , address=45.91.116.6:4973 , piece_id=RPGLDUJH5RVGTA57XUAUT7VDZHHXANEWDT6BHXUPE6SE3RF3FGSA, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.753210772 (24h35m0.215303255s) [noise=true , address=45.91.116.5:4299 , piece_id=3RBI4K5PYX6SHMMGWKX4NGOKVGL2MMSSZK3OOPO7O6ASTEKMVCNA, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.795921140 (24h35m0.258013623s) [noise=true , address=45.91.116.8:4047 , piece_id=4VEYC72WDBDFYFCXD5C5YSCYFNLCCG65F54ENBAOCJHDNNGUOYKQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.826662107 (24h35m0.28875459s) [noise=true , address=77.90.31.15:4698 , piece_id=3Z5SNODVDVEBDNLJIBQY6W2UUL6C253EFLOHI4QQSBQG3REHLANQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.844752764 (24h35m0.306845247s) [noise=true , address=45.91.116.9:4262 , piece_id=IQAHEVE2REPXG3YRUBCSNRNFMOGRZSEW36GQF4BRBS6SLNKLVXUA, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:14.937968625 (24h35m0.400061108s) [noise=true , address=77.90.31.10:4518 , piece_id=O2IDUCHQLU6PAY2YE25HQ477QWECQXFFYHRCOTKX2X5NXNKE326Q, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:15.439818927 (24h35m0.90191141s) [noise=true , address=45.91.116.11:4119 , piece_id=SSMVXKOZJVM25DEKWO5LGID6PXM5MC72YNEHDOSTWBDXRPJOIJQQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: <nil> Sep 24 14:06:41.467470671 (24h35m26.929563154s) [noise=true , address=145.79.227.15:4874 , piece_id=XHH6XEHPS5UAKYV4GBOGSW5H4PNPRRTRK23UAZVQ5JBJQMNVVVHA, seg_pos={0 0}, upload=21642]: Done uploading piece. err: ecclient: upload failed (node:12ay3ZqUvNcaX19JYT1hEaqydsxP6VoDjPavWXBNB34cestcm71, address:145.79.227.15:4874): protocol: write tcp 188.166.225.170:60314->145.79.227.15:4874: use of closed network connection; write tcp 188.166.225.170:60314->145.79.227.15:4874: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:60314->145.79.227.15:4874: use of closed network connection Sep 24 14:06:41.467561505 (24h35m26.929653998s) [noise=true , address=145.79.227.5:4223 , piece_id=YULOT23Y5JT3YRRGKXKPIGRBH4ZCATDQDNZZMYSTZ64XNJ7RVNYQ, seg_pos={0 0}, upload=21642]: Done uploading piece. err: ecclient: upload failed (node:12C59b8knr62ANoUWyaJZDHm2os1tFm3SKL8d34eKsdpZGHYQWE, address:145.79.227.5:4223): protocol: write tcp 188.166.225.170:47376->145.79.227.5:4223: use of closed network connection; write tcp 188.166.225.170:47376->145.79.227.5:4223: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:47376->145.79.227.5:4223: use of closed network connection Sep 24 14:06:41.467579378 (24h35m26.929671881s) [noise=true , address=145.79.227.6:4258 , piece_id=7DH7B3UOTJ5LU5PSEDA23P7NZBVMCQ4MMI7NSRP4VA2VD2L32C4Q, seg_pos={0 0}, upload=21642]: Done uploading piece. err: ecclient: upload failed (node:12G8RkyjiDQ2nNkM8drWhAKuBQfGizv711hyBDNB11noDzq1xtF, address:145.79.227.6:4258): protocol: write tcp 188.166.225.170:42296->145.79.227.6:4258: use of closed network connection; write tcp 188.166.225.170:42296->145.79.227.6:4258: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:42296->145.79.227.6:4258: use of closed network connection Sep 24 14:06:41.467652768 (24h35m26.929745251s) [noise=true , address=145.79.227.13:4801 , piece_id=KNS7UF4X4WB55YP7EZAR6XLU2C4WKIMUTFMPFGRTBO7IJNOEJ46A, seg_pos={0 0}, upload=21642]: Done uploading piece. err: ecclient: upload failed (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ, address:145.79.227.13:4801): protocol: write tcp 188.166.225.170:36514->145.79.227.13:4801: use of closed network connection; write tcp 188.166.225.170:36514->145.79.227.13:4801: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:36514->145.79.227.13:4801: use of closed network connection Sep 24 14:06:41.467673277 (24h35m26.92976576s) [seg_pos={0 0}, upload=21642]: Done waiting for segment. successful: 12 optimal: 13 errs: <nil> Sep 24 14:06:41.706187501 (24h35m27.168279984s) [upload=21642]: Done uploading segments... Sep 24 14:06:41.706248277 (24h35m27.16834077s) [upload=21642]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 12) Duration: 30.241362058s core uplink us-select-1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 762.199126ms target: https://us1.storj.io/login FAIL (2025-09-24 14:05:29 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 762.228864ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 688.774642ms target: https://us1.storj.io/login FAIL (2025-09-24 14:04:43 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 688.793177ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..X.........................................................] duration: 702.839563ms target: https://us1.storj.io/login FAIL (2025-09-24 14:04:03 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 702.893967ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...X........................................................] duration: 1.113056409s target: https://us1.storj.io/login FAIL (2025-09-24 14:03:30 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 1.113085455s core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....X.......................................................] duration: 740.859611ms target: https://us1.storj.io/login FAIL (2025-09-24 14:02:59 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 740.889828ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....X......................................................] duration: 740.54428ms target: https://us1.storj.io/login FAIL (2025-09-24 14:02:41 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 740.56969ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......X.....................................................] duration: 1.992662161s target: https://us1.storj.io/login FAIL (2025-09-24 14:01:33 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 1.992687729s core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......X....................................................] duration: 809.101416ms target: https://us1.storj.io/login FAIL (2025-09-24 14:01:06 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 809.420513ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........X...................................................] duration: 756.490856ms target: https://us1.storj.io/login FAIL (2025-09-24 14:00:39 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 756.545181ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........X..................................................] duration: 1.22662189s target: https://us1.storj.io/login FAIL (2025-09-24 14:00:16 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 1.226656586s core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........X.................................................] duration: 772.864687ms target: https://us1.storj.io/login FAIL (2025-09-24 13:59:10 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 772.89293ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........X................................................] duration: 702.773288ms target: https://us1.storj.io/login FAIL (2025-09-24 13:58:03 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 702.80111ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............X...............................................] duration: 697.586973ms target: https://us1.storj.io/login FAIL (2025-09-24 13:57:36 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 697.625806ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............X..............................................] duration: 677.012836ms target: https://us1.storj.io/login FAIL (2025-09-24 13:56:32 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 677.032023ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............X.............................................] duration: 676.754623ms target: https://us1.storj.io/login FAIL (2025-09-24 13:56:09 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 676.782477ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............X............................................] duration: 785.70913ms target: https://us1.storj.io/login FAIL (2025-09-24 13:55:19 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 785.739439ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................X...........................................] duration: 735.93825ms target: https://us1.storj.io/login FAIL (2025-09-24 13:54:20 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 735.961553ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................X..........................................] duration: 727.174222ms target: https://us1.storj.io/login FAIL (2025-09-24 13:53:48 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 727.223917ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................X.........................................] duration: 1.066220928s target: https://us1.storj.io/login FAIL (2025-09-24 13:53:26 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 1.066265814s core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...................X........................................] duration: 681.046718ms target: https://us1.storj.io/login FAIL (2025-09-24 13:52:25 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 681.081052ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [....................X.......................................] duration: 684.705463ms target: https://us1.storj.io/login FAIL (2025-09-24 13:51:43 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 684.766439ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.....................X......................................] duration: 695.661947ms target: https://us1.storj.io/login FAIL (2025-09-24 13:51:13 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 695.690773ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [......................X.....................................] duration: 750.890444ms target: https://us1.storj.io/login FAIL (2025-09-24 13:50:08 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 750.929388ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.......................X....................................] duration: 787.921275ms target: https://us1.storj.io/login FAIL (2025-09-24 13:48:56 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 787.949828ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [........................X...................................] duration: 748.15161ms target: https://us1.storj.io/login FAIL (2025-09-24 13:48:07 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 748.195724ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.........................X..................................] duration: 738.381295ms target: https://us1.storj.io/login FAIL (2025-09-24 13:47:01 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 738.416422ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..........................X.................................] duration: 791.888392ms target: https://us1.storj.io/login FAIL (2025-09-24 13:46:02 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 791.923128ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...........................X................................] duration: 740.803407ms target: https://us1.storj.io/login FAIL (2025-09-24 13:45:13 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 740.844486ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [............................X...............................] duration: 741.662894ms target: https://us1.storj.io/login FAIL (2025-09-24 13:44:02 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 741.734952ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.............................X..............................] duration: 747.17482ms target: https://us1.storj.io/login FAIL (2025-09-24 13:43:42 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 747.214235ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..............................X.............................] duration: 1.392723385s target: https://us1.storj.io/login FAIL (2025-09-24 13:42:32 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 1.392752549s core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [...............................X............................] duration: 765.084643ms target: https://us1.storj.io/login FAIL (2025-09-24 13:42:04 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 765.113236ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [................................X...........................] duration: 730.593215ms target: https://us1.storj.io/login FAIL (2025-09-24 13:41:47 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 730.654581ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [.................................X..........................] duration: 737.85313ms target: https://us1.storj.io/login FAIL (2025-09-24 13:41:02 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 737.88986ms core satellite us1
checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login) antiblip: 1 out of 60 is failed (threshold=3) [..................................X.........................] duration: 791.054204ms target: https://us1.storj.io/login FAIL (2025-09-24 13:39:54 UTC) orbiter warning: Get "https://us1.storj.io/login": dial tcp 35.212.10.183:443: connect: connection refused Duration: 791.08311ms core satellite us1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)

checkers.HTTP{"GET" "https://us1.storj.io/login", TLSCertExpirationWarning: 672h0m0s, TLSCertExpirationError: 336h0m0s} failure (antiblip window 3/60): (http.us1.satellite_login)