Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............XX..............................................] duration: 5.835326411s FAIL (2025-07-19 17:04:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.835343986s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............XX.............................................] duration: 5.712713022s FAIL (2025-07-19 17:04:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.712740323s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............XX............................................] duration: 5.757101262s FAIL (2025-07-19 17:03:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.757117292s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............XX...........................................] duration: 5.836524363s FAIL (2025-07-19 17:01:56 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.836545722s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................XX..........................................] duration: 5.815686893s FAIL (2025-07-19 17:01:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.815717583s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................XX.........................................] duration: 5.808344857s FAIL (2025-07-19 17:01:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.808373052s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................XX........................................] duration: 5.921357313s FAIL (2025-07-19 17:00:16 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.921382522s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................XX.......................................] duration: 5.7338172s FAIL (2025-07-19 16:59:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.733846105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................XX......................................] duration: 5.799880561s FAIL (2025-07-19 16:58:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.799906269s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................XX.....................................] duration: 5.755719967s FAIL (2025-07-19 16:57:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.755742381s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................XX....................................] duration: 6.004825787s FAIL (2025-07-19 16:57:04 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.004849914s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................XX...................................] duration: 5.989878879s FAIL (2025-07-19 16:56:00 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.989954614s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................XX..................................] duration: 5.759859161s FAIL (2025-07-19 16:55:37 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.759889109s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................XX.................................] duration: 5.966478213s FAIL (2025-07-19 16:54:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.966499684s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................XX................................] duration: 5.824749114s FAIL (2025-07-19 16:54:03 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.82476278s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................XX...............................] duration: 5.90950013s FAIL (2025-07-19 16:53:46 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.909603037s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................XX..............................] duration: 6.002351581s FAIL (2025-07-19 16:53:09 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 6.002375337s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................XX.............................] duration: 5.912046399s FAIL (2025-07-19 16:51:58 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.912082768s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................XX............................] duration: 5.799244072s FAIL (2025-07-19 16:51:05 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.799263109s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................XX...........................] duration: 5.793482049s FAIL (2025-07-19 16:50:01 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.793506825s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................XX..........................] duration: 5.798176091s FAIL (2025-07-19 16:49:36 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.798200818s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................XX.........................] duration: 5.718158402s FAIL (2025-07-19 16:48:41 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.718184361s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................XX........................] duration: 5.858521789s FAIL (2025-07-19 16:48:22 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.858547597s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................XX.......................] duration: 5.878297342s FAIL (2025-07-19 16:47:59 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.878327399s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................XX......................] duration: 5.871943393s FAIL (2025-07-19 16:47:23 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.87196782s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................XX.....................] duration: 5.753008211s FAIL (2025-07-19 16:46:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.753037466s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................XX....................] duration: 5.818892873s FAIL (2025-07-19 16:45:38 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.81891728s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................XX...................] duration: 5.672343141s FAIL (2025-07-19 16:44:24 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.672372388s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................XX..................] duration: 5.858868023s FAIL (2025-07-19 16:43:20 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.858893612s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................XX.................] duration: 5.868753341s FAIL (2025-07-19 16:42:28 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.86877904s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................XX................] duration: 5.784267008s FAIL (2025-07-19 16:42:10 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.784291115s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...........................................XX...............] duration: 5.85307601s FAIL (2025-07-19 16:41:31 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.853102161s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [............................................XX..............] duration: 5.700121231s FAIL (2025-07-19 16:41:06 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.700144857s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.............................................XX.............] duration: 5.843737223s FAIL (2025-07-19 16:40:17 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.843765957s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..............................................XX............] duration: 5.915424064s FAIL (2025-07-19 16:39:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.915446577s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...............................................XX...........] duration: 5.870483089s FAIL (2025-07-19 16:39:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.870510542s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [................................................XX..........] duration: 5.944358832s FAIL (2025-07-19 16:38:57 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.944386153s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.................................................XX.........] duration: 5.812672244s FAIL (2025-07-19 16:37:53 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.812701068s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..................................................XX........] duration: 5.743398282s FAIL (2025-07-19 16:37:21 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.743419233s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [...................................................XX.......] duration: 5.955722742s FAIL (2025-07-19 16:36:44 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.955749493s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [....................................................XX......] duration: 5.867163805s FAIL (2025-07-19 16:36:08 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.867179847s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.....................................................XX.....] duration: 5.871671421s FAIL (2025-07-19 16:35:13 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.871710184s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [......................................................XX....] duration: 5.890275425s FAIL (2025-07-19 16:34:07 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.890306876s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.......................................................XX...] duration: 5.850446199s FAIL (2025-07-19 16:33:19 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.850468652s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [........................................................XX..] duration: 5.968269711s FAIL (2025-07-19 16:32:40 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.968293105s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [.........................................................XX.] duration: 5.858034714s FAIL (2025-07-19 16:31:42 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 5.858068458s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [..........................................................XX] duration: 30.264037651s FAIL (2025-07-19 16:31:34 UTC) orbiter warning: ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 6) ======LOGS====== 8s) [upload=14056]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:31:05.338046739 (17h40m11.710626155s) [upload=14056]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:31:05.576786510 (17h40m11.949365895s) [seg_pos={0 0}, upload=14056]: Begin upload segment... Jul 19 16:31:05.576839321 (17h40m11.949418686s) [seg_pos={0 0}, upload=14056]: Done begin upload segment. Jul 19 16:31:05.576858887 (17h40m11.949438242s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576868266 (17h40m11.949447621s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.576911679 (17h40m11.949491034s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577162077 (17h40m11.949741433s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577345018 (17h40m11.949924373s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577359566 (17h40m11.949938921s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577572773 (17h40m11.950152128s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.577853591 (17h40m11.950432946s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578146221 (17h40m11.950725576s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578469920 (17h40m11.951049275s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578781686 (17h40m11.951361041s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.578983102 (17h40m11.951562457s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579188916 (17h40m11.951768271s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579397766 (17h40m11.951977121s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579600513 (17h40m11.952179868s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:05.579789265 (17h40m11.95236861s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Uploading piece... Jul 19 16:31:07.418281321 (17h40m13.790860676s) [upload=14056]: Next returned nil segment Jul 19 16:31:07.418298594 (17h40m13.790877949s) [upload=14056]: Waiting for error group managing segments... Jul 19 16:31:07.713732460 (17h40m14.086311815s) [noise=true , address=104.36.82.13:4186 , piece_id=AMP5YR3FNDWFV4LKI2QXWW4LROBQZ7GJHPX62ZJGZHJXKGKRLWDQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725430082 (17h40m14.098009427s) [noise=true , address=104.36.82.6:4875 , piece_id=KYJZRJ4PRBUPPRSTSU7EIDG3W6GH4VHJFGGQDG3O6RDJUCXC7NKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.725691211 (17h40m14.098270566s) [noise=true , address=104.36.82.8:4971 , piece_id=F4REAZ3LEGONDBBCPIXZ7PZGN4UVL3HQVXO6OOFOTSC6T4Z4U24A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:07.800874934 (17h40m14.173454289s) [noise=true , address=104.36.82.16:4295 , piece_id=D2NKHYZOMXW7QGT62KJXK4CKUG6QK4ENC427BVHGJJ5GDST26OWQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:10.093208392 (17h40m16.465787737s) [noise=true , address=104.36.82.11:4113 , piece_id=IMJ3K7333JAJSH52PWZLH4WOCS4Q3HQBEIXXLJCMWDUHEEHR7STA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:32.295856427 (17h40m38.668435782s) [noise=true , address=45.91.116.10:4083 , piece_id=3BMFSZ4PKNA7YJFV5RIJZ6R6TPWKBWCTIPBWFNJ4KOCTXNHQ7WXQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: <nil> Jul 19 16:31:34.694467750 (17h40m41.067047095s) [noise=true , address=77.90.31.16:4735 , piece_id=RZTPQEIZ5VGEHKPYFSLYO6FJGN5Q2JLTBLDQPDR22P5B5ODHGMOQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:14LxqUFh6udxGJH9qJV6d1tdGgxyzv3qiZYENLtXffBCpLeG9h, address:77.90.31.16:4735): protocol: rpc: dial tcp 77.90.31.16:4735: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694564936 (17h40m41.067144291s) [noise=true , address=77.90.31.15:4698 , piece_id=XTDJBOYI4EA7NP55AF2KBDBNWJPKN7UWLOCWPZOVQXCNPCBYCW3Q, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12XVNxzmNrzdDjyJR5e6M394ptZY4sMPdH1suN42y5WynbVp8oj, address:77.90.31.15:4698): protocol: write tcp 188.166.225.170:39470->77.90.31.15:4698: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694621445 (17h40m41.06720081s) [noise=true , address=77.90.31.15:4699 , piece_id=FHCFBZXFDJU4CCYMH7GHRNL5OOPIPTIXZ56WEVVDXXOA3ONISUMQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Rb4DbV6syjrfdRBUUZQ4efA3xtwDFawUFp5GdQRc4GaDS2T8M, address:77.90.31.15:4699): protocol: rpc: dial tcp 77.90.31.15:4699: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694670959 (17h40m41.067250314s) [noise=true , address=45.91.116.6:4973 , piece_id=FUII5FH6ATQGBQTHZT7FM5FSFEWUUWBKYED7CJF4JO2SX3ILCI6A, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12Z23fYAQ6w3Wv3mXVfoqQcLR7HxW5yhrPe2xgViVeiKFsmRVcV, address:45.91.116.6:4973): protocol: context deadline exceeded; context deadline exceeded Jul 19 16:31:34.694778735 (17h40m41.06735809s) [noise=true , address=45.91.116.4:4939 , piece_id=JYDVNUMRVWLVX6N7LFG7HXLK2AJ7EDPVU3D7P747TVWBHRYJXYUQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12K4acSz27pP9v6DZSvETPuUfzFFiR2k7U3awntw2eHNytnPiuX, address:45.91.116.4:4939): protocol: rpc: dial tcp 45.91.116.4:4939: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694831015 (17h40m41.06741037s) [noise=true , address=77.90.31.19:4843 , piece_id=MLPAW2UQCLX2X3MYR5TRFC2WA3DGQTY7BNJMJ5ZC73C3LMXTAOVQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12akCPrMTihqq6CKv7VtDbHhXnWrXygXXhZKgZbftGbymRNzuMb, address:77.90.31.19:4843): protocol: rpc: dial tcp 77.90.31.19:4843: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694871242 (17h40m41.067450588s) [noise=true , address=77.90.31.14:4662 , piece_id=4IABFU5ALTR5N3HRIKIRX3HKXH6P6AGQ74VIWVOYSQO42MTLWUKQ, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:138doLw3dLzEZXoSRNbCridRmSCajLgdVZ86vBdAQuWXjaDD8n, address:77.90.31.14:4662): protocol: rpc: dial tcp 77.90.31.14:4662: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694904406 (17h40m41.067483761s) [noise=true , address=45.91.116.14:4191 , piece_id=UM6JNWNZDMABIIDTQGSHZGQX4SAFCGKIWWDVZN43ANHJPVX6EWOA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1FuLM1f14cPdZhV3wWmYD746WFCn1jMHjHyaK7s7fSomud39Zq, address:45.91.116.14:4191): protocol: rpc: dial tcp 45.91.116.14:4191: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694939703 (17h40m41.067519058s) [noise=true , address=77.90.31.21:4877 , piece_id=B2V2C2NOBAT56IZAFATVUTAUVHKDMZHYVBTOLN76YQTC2XPJUJDA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:12qy6swRPui25KAsuj9PzHvijxQFHoaLMJ95LvK34pCA9UU3gR2, address:77.90.31.21:4877): protocol: rpc: dial tcp 77.90.31.21:4877: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694973868 (17h40m41.067553223s) [noise=true , address=77.90.31.18:4806 , piece_id=O2NBK436OUJHWCMBE5ZDT4ICQ5BDCZOMDIHUSN2FFUEE6QLDSCTA, seg_pos={0 0}, upload=14056]: Done uploading piece. err: ecclient: upload failed (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ, address:77.90.31.18:4806): protocol: write tcp 188.166.225.170:33688->77.90.31.18:4806: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:31:34.694988497 (17h40m41.067567852s) [seg_pos={0 0}, upload=14056]: Done waiting for segment. successful: 6 optimal: 13 errs: <nil> Jul 19 16:31:34.955999941 (17h40m41.328579296s) [upload=14056]: Done uploading segments... Jul 19 16:31:34.956058423 (17h40m41.328637778s) [upload=14056]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 6) Duration: 30.264063961s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [...........................................................X] duration: 30.002081833s FAIL (2025-07-19 16:30:47 UTC) orbiter warning: ======ERROR====== uplink: context deadline exceeded ======LOGS====== =7GGPCKCL4PZ6UBQME6IC3ADQD4HXBDHJZNXRU7UVCDEQAAJFEREQ, seg_pos={0 0}, upload=14037]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: expected piece hash; context canceled; EOF Jul 19 16:29:26.761455771 (17h38m33.134035126s) [seg_pos={0 0}, upload=14037]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:29:26.761507510 (17h38m33.134086865s) [upload=14037]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:29:26.761511087 (17h38m33.134090432s) [upload=14037]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:29:27.107178687 (17h38m33.479758042s) [upload=14037]: Done uploading segments... Jul 19 16:29:27.107195199 (17h38m33.479774554s) [upload=14037]: Upload finished. err: <nil> Jul 19 16:30:18.262231002 (17h39m24.634810347s) [upload=14047]: Starting upload Jul 19 16:30:18.262293051 (17h39m24.634872406s) [upload=14047]: Done starting upload Jul 19 16:30:18.262357545 (17h39m24.6349369s) [upload=14047]: Uploading segments... Jul 19 16:30:18.262544713 (17h39m24.635124068s) [upload=14047]: Got next segment. Inline: false Jul 19 16:30:18.262575542 (17h39m24.635154897s) [upload=14047]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 16:30:18.262581092 (17h39m24.635160437s) [upload=14047]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 16:30:18.533099406 (17h39m24.905678761s) [seg_pos={0 0}, upload=14047]: Begin upload segment... Jul 19 16:30:18.533141446 (17h39m24.905720801s) [seg_pos={0 0}, upload=14047]: Done begin upload segment. Jul 19 16:30:18.533164731 (17h39m24.905744086s) [noise=true , address=45.91.116.6:4975 , piece_id=WCNVN6XSP3ZER5YWIZSTR6TENHRPPBBOJWOCR4PH5TPK5XIVXOXQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.533547714 (17h39m24.906127069s) [noise=true , address=104.36.82.16:4295 , piece_id=FTLDSAIP44NSQQKIZU5Y54BOHB7UVMSTKEHF2BTRL2ST325664RQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.533839442 (17h39m24.906418787s) [noise=true , address=77.90.31.12:4590 , piece_id=KH5OMXWQQUBHSZAM775FBFBTE6LAUVS2VEEZGVDC5G4NMO5HSZJA, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.534092727 (17h39m24.906672082s) [noise=true , address=45.91.116.9:4261 , piece_id=GXQKQUW266MBANVQ7533IAP6BHD3X3T5XUESURT5GCX6J2DH5I7A, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.534317066 (17h39m24.906896421s) [noise=true , address=104.36.82.5:4840 , piece_id=TBU67HBAPO6OCM4CDKNWJ6JIG2N5V76J4JCFUOWXTSNPQK2UPLIA, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.534537157 (17h39m24.907116512s) [noise=true , address=104.36.82.14:4223 , piece_id=J33LU6ZDQK6WPSJSDBM35FMBD5VS7LYVJPVX7Y3PNCG6SRX2BJ7Q, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.534754202 (17h39m24.907333558s) [noise=true , address=77.90.31.10:4520 , piece_id=EULG4TZPFZJFE57XXV7QK25FRZIEZOCH6UJC562MZJZM7TNEH73A, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.534970216 (17h39m24.907549561s) [noise=true , address=104.36.82.3:4767 , piece_id=5UJ4QYVOOHLE6P4MHWRYSOIZ6R25A2SAT6QKWPORKDXQJ4RCXF5Q, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.535185026 (17h39m24.907764381s) [noise=true , address=45.91.116.5:4297 , piece_id=VJ6INY6T625A2OCC3SXERXJGGO37REU5M7XZ5EXM37GDUQZXIPWA, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.535368598 (17h39m24.907947943s) [noise=true , address=77.90.31.11:4555 , piece_id=S27F4ILK7RG4WC7TF43FIJ4O2R2M54QXQL5T56HVP5HHLII3DGSA, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.535547290 (17h39m24.908126645s) [noise=true , address=45.91.116.12:4226 , piece_id=XPUQCCCBJBUEWHU6HZHVSZLX4AGE623ATHA435VPP5JMFKNLP2DQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.535723978 (17h39m24.908303333s) [noise=true , address=45.91.116.10:4082 , piece_id=EYTHEXNU4CVMUA653VX44CRV74YC36EDXGMXEPDCTMWEAHU6MEGQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.535914242 (17h39m24.908493597s) [noise=true , address=104.36.82.15:4259 , piece_id=LE3VUUOFUKYVZG2HM56ZZHBFOYEZX2I2TD2TERJIVSTY45RMZSYQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.536130145 (17h39m24.9087095s) [noise=true , address=45.91.116.4:4940 , piece_id=DFEG4XGIMGEYPEUUUIJSNPVIUF7NTWRTKRSZYG7573HGNADEV7PQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.536324918 (17h39m24.908904263s) [noise=true , address=45.91.116.7:4010 , piece_id=CPCKQD3WU3WYYCYZXKSNQOE4B3XCF3FON7ZUOB7YWMJZZY3BCQUQ, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:18.536508189 (17h39m24.909087544s) [noise=true , address=77.90.31.18:4806 , piece_id=Z72ZHLQTSEKCROFYMWABO7LSUSKYN5CZQXIAM7EZPDGKOGKIZM5A, seg_pos={0 0}, upload=14047]: Uploading piece... Jul 19 16:30:20.295444134 (17h39m26.668023489s) [upload=14047]: Next returned nil segment Jul 19 16:30:20.295459825 (17h39m26.66803918s) [upload=14047]: Waiting for error group managing segments... Jul 19 16:30:20.585926255 (17h39m26.95850564s) [noise=true , address=45.91.116.12:4226 , piece_id=XPUQCCCBJBUEWHU6HZHVSZLX4AGE623ATHA435VPP5JMFKNLP2DQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.594393347 (17h39m26.966972702s) [noise=true , address=45.91.116.9:4261 , piece_id=GXQKQUW266MBANVQ7533IAP6BHD3X3T5XUESURT5GCX6J2DH5I7A, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.605066368 (17h39m26.977645723s) [noise=true , address=45.91.116.7:4010 , piece_id=CPCKQD3WU3WYYCYZXKSNQOE4B3XCF3FON7ZUOB7YWMJZZY3BCQUQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.651859228 (17h39m27.024438583s) [noise=true , address=104.36.82.16:4295 , piece_id=FTLDSAIP44NSQQKIZU5Y54BOHB7UVMSTKEHF2BTRL2ST325664RQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.659494156 (17h39m27.032073511s) [noise=true , address=104.36.82.5:4840 , piece_id=TBU67HBAPO6OCM4CDKNWJ6JIG2N5V76J4JCFUOWXTSNPQK2UPLIA, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.675476246 (17h39m27.048055601s) [noise=true , address=104.36.82.14:4223 , piece_id=J33LU6ZDQK6WPSJSDBM35FMBD5VS7LYVJPVX7Y3PNCG6SRX2BJ7Q, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.676362661 (17h39m27.048942016s) [noise=true , address=104.36.82.15:4259 , piece_id=LE3VUUOFUKYVZG2HM56ZZHBFOYEZX2I2TD2TERJIVSTY45RMZSYQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.680788691 (17h39m27.053368046s) [noise=true , address=45.91.116.4:4940 , piece_id=DFEG4XGIMGEYPEUUUIJSNPVIUF7NTWRTKRSZYG7573HGNADEV7PQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.721891895 (17h39m27.09447125s) [noise=true , address=104.36.82.3:4767 , piece_id=5UJ4QYVOOHLE6P4MHWRYSOIZ6R25A2SAT6QKWPORKDXQJ4RCXF5Q, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.721999671 (17h39m27.094579026s) [noise=true , address=45.91.116.6:4975 , piece_id=WCNVN6XSP3ZER5YWIZSTR6TENHRPPBBOJWOCR4PH5TPK5XIVXOXQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.786329776 (17h39m27.158909121s) [noise=true , address=45.91.116.5:4297 , piece_id=VJ6INY6T625A2OCC3SXERXJGGO37REU5M7XZ5EXM37GDUQZXIPWA, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:20.822790408 (17h39m27.195369763s) [noise=true , address=45.91.116.10:4082 , piece_id=EYTHEXNU4CVMUA653VX44CRV74YC36EDXGMXEPDCTMWEAHU6MEGQ, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:46.719153036 (17h39m53.091732381s) [noise=true , address=77.90.31.10:4520 , piece_id=EULG4TZPFZJFE57XXV7QK25FRZIEZOCH6UJC562MZJZM7TNEH73A, seg_pos={0 0}, upload=14047]: Done uploading piece. err: <nil> Jul 19 16:30:46.719180218 (17h39m53.091759573s) [seg_pos={0 0}, upload=14047]: Segment reached optimal threshold of 13 pieces. Jul 19 16:30:46.719376964 (17h39m53.09195633s) [noise=true , address=77.90.31.18:4806 , piece_id=Z72ZHLQTSEKCROFYMWABO7LSUSKYN5CZQXIAM7EZPDGKOGKIZM5A, seg_pos={0 0}, upload=14047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1gJNAhKazJp5TabeEkXsEUFjoBu87mfYDBdeKhfGi4gub3yaSJ): protocol: context canceled; context canceled Jul 19 16:30:46.719494210 (17h39m53.092073565s) [noise=true , address=77.90.31.12:4590 , piece_id=KH5OMXWQQUBHSZAM775FBFBTE6LAUVS2VEEZGVDC5G4NMO5HSZJA, seg_pos={0 0}, upload=14047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1i9B5aDDfPkgveDvLh7ngsnCJb1Viw2cHYNMmgbjhG5BkWFCMw): protocol: read tcp 188.166.225.170:58014->77.90.31.12:4590: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:30:46.719556769 (17h39m53.092136124s) [noise=true , address=77.90.31.11:4555 , piece_id=S27F4ILK7RG4WC7TF43FIJ4O2R2M54QXQL5T56HVP5HHLII3DGSA, seg_pos={0 0}, upload=14047]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1aRnwUHwyB11FpSDzQWDikCaEu6YCnvKwy9VRFYq8sbm7K6jgQ): protocol: write tcp 188.166.225.170:60692->77.90.31.11:4555: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Jul 19 16:30:46.719578240 (17h39m53.092157595s) [seg_pos={0 0}, upload=14047]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 16:30:46.719644607 (17h39m53.092223952s) [upload=14047]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 16:30:46.719648444 (17h39m53.092227799s) [upload=14047]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 16:30:47.023375408 (17h39m53.395954763s) [upload=14047]: Done uploading segments... Jul 19 16:30:47.023394544 (17h39m53.395973899s) [upload=14047]: Upload finished. err: <nil> Duration: 30.002099656s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 5.664275082s FAIL (2025-07-19 07:43:01 UTC) orbiter warning: ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== ARML7R2U7MK5UBOQLTMNEEL2KMPQ, seg_pos={0 0}, upload=6394]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:01.174242029 (8h5m7.546821384s) [noise=true , address=104.36.82.10:4079 , piece_id=CQZTVCO3IEWAGADQCO7PUGO3ZYPMQXKHZ2UPR4X3WXR4UOXD23DQ, seg_pos={0 0}, upload=6394]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:01.174255615 (8h5m7.54683497s) [seg_pos={0 0}, upload=6394]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 06:56:01.174325238 (8h5m7.546904634s) [upload=6394]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 06:56:01.174333464 (8h5m7.546912819s) [upload=6394]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 06:56:01.474962706 (8h5m7.847542061s) [upload=6394]: Done uploading segments... Jul 19 06:56:01.474982223 (8h5m7.847561578s) [upload=6394]: Upload finished. err: <nil> Jul 19 06:56:48.315803186 (8h5m54.688382541s) [upload=6403]: Starting upload Jul 19 06:56:48.315875724 (8h5m54.688455079s) [upload=6403]: Done starting upload Jul 19 06:56:48.315947812 (8h5m54.688527167s) [upload=6403]: Uploading segments... Jul 19 06:56:48.316122416 (8h5m54.688701761s) [upload=6403]: Got next segment. Inline: false Jul 19 06:56:48.316150149 (8h5m54.688729494s) [upload=6403]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 06:56:48.316155278 (8h5m54.688734633s) [upload=6403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 06:56:48.549320531 (8h5m54.921899886s) [seg_pos={0 0}, upload=6403]: Begin upload segment... Jul 19 06:56:48.549361600 (8h5m54.921940955s) [seg_pos={0 0}, upload=6403]: Done begin upload segment. Jul 19 06:56:48.549382059 (8h5m54.921961404s) [noise=true , address=77.90.31.21:4877 , piece_id=EDLQFKGHOTHTLWL7PPUOMRTA6JIL7TF7HU22JNO2EVVXETFLUHYQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549433146 (8h5m54.922012501s) [noise=true , address=104.36.82.13:4185 , piece_id=JHS6G7M7N5IVY2VPSPHJPVRZQHQJIYBU54KKUW47COZP2YGQ2XGA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549694476 (8h5m54.922273831s) [noise=true , address=45.91.116.5:4299 , piece_id=OHEAHX6PK3JW6K3BYMJQYK4HDNP6K7DEJOW7EOQKK2TC4P6NW4CA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549875533 (8h5m54.922454888s) [noise=true , address=45.91.116.13:4153 , piece_id=MPAR7SWMVQ236Z76HU7FC3LMQ4NRUD3TN64HT3MQRWTWGBQ6YDSQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549937652 (8h5m54.922517007s) [noise=true , address=45.91.116.8:4045 , piece_id=G5QHS6Y4T2JY7PYOXKUGSZM75F4NWF37L45H434N5WOZKEDUMPWA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550155920 (8h5m54.922735275s) [noise=true , address=45.91.116.11:4117 , piece_id=BZQFV37K7LE4Z2K54HPA6PBIVNZEC6SBX4HYUE6YGRCHLYUOOEBA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550269487 (8h5m54.922848832s) [noise=true , address=45.91.116.12:4227 , piece_id=A25PGAY3W24PI2WFW6NYMWVSW3NQK4DC5ZJMC4VISWYQICT3SJQA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550430595 (8h5m54.92300994s) [noise=true , address=104.36.82.15:4259 , piece_id=HAQGLMVTGJXXK334AOVADV45LX4MBWBSDOUHZ7HXYB4CZJ6SWM5A, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550495319 (8h5m54.923074664s) [noise=true , address=77.90.31.19:4841 , piece_id=WJMRBXWLIAPBM4YW6SZDQM3A536PDDRMOJF5R4J2LVRDVJPULP4A, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550699370 (8h5m54.923278715s) [noise=true , address=45.91.116.4:4938 , piece_id=ZIJIIBKIGQY2BRI3DVAEWWLPPPUDB2RYMYV2EAD3EZLO2K2P2WTA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550722122 (8h5m54.923301477s) [noise=true , address=77.90.31.15:4698 , piece_id=6PREKXY6754B6THCWIZRYR3AO3S2XGPR4GSI7WQ2UNJW2NDWAYEQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550951211 (8h5m54.923530566s) [noise=true , address=77.90.31.10:4520 , piece_id=PV2J4IO3YYWYVEKXTS4C2CT3BIFNDJNJOYCYDQC5LQC2B4TZCU3A, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550955099 (8h5m54.923534444s) [noise=true , address=104.36.82.2:4731 , piece_id=MIOJJFTFCRXMSFE4FU2AKBFTPVIX5JEGOJV5FIX5OGUEUV2OSW7Q, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.551183094 (8h5m54.923762439s) [noise=true , address=77.90.31.12:4589 , piece_id=AMXTBSD6WO4UEXIIXO6L5ZQDD3AYUZ75PIMNMX6CXQTPAUS6MKNQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.551205177 (8h5m54.923784522s) [noise=true , address=104.36.82.14:4222 , piece_id=S2E5GVHNN4AVBWUCQ7ZF4VTBMCE44VKD3ONII57T52EQAGAQBEBQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.551419828 (8h5m54.923999173s) [noise=true , address=45.91.116.10:4083 , piece_id=4AIGZG7ABSINNQYFQLPC3ZKRGRD7LUZ6AP4ARHVFHOCYPJY3NPNA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:50.313191049 (8h5m56.685770404s) [upload=6403]: Next returned nil segment Jul 19 06:56:50.313210367 (8h5m56.685789712s) [upload=6403]: Waiting for error group managing segments... Jul 19 06:56:50.602094226 (8h5m56.974673571s) [noise=true , address=45.91.116.4:4938 , piece_id=ZIJIIBKIGQY2BRI3DVAEWWLPPPUDB2RYMYV2EAD3EZLO2K2P2WTA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.606309361 (8h5m56.978888716s) [noise=true , address=104.36.82.14:4222 , piece_id=S2E5GVHNN4AVBWUCQ7ZF4VTBMCE44VKD3ONII57T52EQAGAQBEBQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.610631160 (8h5m56.983210515s) [noise=true , address=45.91.116.5:4299 , piece_id=OHEAHX6PK3JW6K3BYMJQYK4HDNP6K7DEJOW7EOQKK2TC4P6NW4CA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.613288184 (8h5m56.985867539s) [noise=true , address=45.91.116.8:4045 , piece_id=G5QHS6Y4T2JY7PYOXKUGSZM75F4NWF37L45H434N5WOZKEDUMPWA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.624484997 (8h5m56.997064352s) [noise=true , address=45.91.116.11:4117 , piece_id=BZQFV37K7LE4Z2K54HPA6PBIVNZEC6SBX4HYUE6YGRCHLYUOOEBA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.626515311 (8h5m56.999094656s) [noise=true , address=77.90.31.21:4877 , piece_id=EDLQFKGHOTHTLWL7PPUOMRTA6JIL7TF7HU22JNO2EVVXETFLUHYQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.627277630 (8h5m56.999856975s) [noise=true , address=77.90.31.19:4841 , piece_id=WJMRBXWLIAPBM4YW6SZDQM3A536PDDRMOJF5R4J2LVRDVJPULP4A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.642423930 (8h5m57.015003285s) [noise=true , address=77.90.31.15:4698 , piece_id=6PREKXY6754B6THCWIZRYR3AO3S2XGPR4GSI7WQ2UNJW2NDWAYEQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.644061784 (8h5m57.016641129s) [noise=true , address=77.90.31.12:4589 , piece_id=AMXTBSD6WO4UEXIIXO6L5ZQDD3AYUZ75PIMNMX6CXQTPAUS6MKNQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.647067965 (8h5m57.01964732s) [noise=true , address=77.90.31.10:4520 , piece_id=PV2J4IO3YYWYVEKXTS4C2CT3BIFNDJNJOYCYDQC5LQC2B4TZCU3A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.678272179 (8h5m57.050851525s) [noise=true , address=104.36.82.13:4185 , piece_id=JHS6G7M7N5IVY2VPSPHJPVRZQHQJIYBU54KKUW47COZP2YGQ2XGA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.684092075 (8h5m57.05667143s) [noise=true , address=45.91.116.13:4153 , piece_id=MPAR7SWMVQ236Z76HU7FC3LMQ4NRUD3TN64HT3MQRWTWGBQ6YDSQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.701086190 (8h5m57.073665535s) [noise=true , address=104.36.82.15:4259 , piece_id=HAQGLMVTGJXXK334AOVADV45LX4MBWBSDOUHZ7HXYB4CZJ6SWM5A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.701112961 (8h5m57.073692306s) [seg_pos={0 0}, upload=6403]: Segment reached optimal threshold of 13 pieces. Jul 19 06:56:50.701241187 (8h5m57.073820542s) [noise=true , address=104.36.82.2:4731 , piece_id=MIOJJFTFCRXMSFE4FU2AKBFTPVIX5JEGOJV5FIX5OGUEUV2OSW7Q, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:50.701311190 (8h5m57.073890545s) [noise=true , address=45.91.116.10:4083 , piece_id=4AIGZG7ABSINNQYFQLPC3ZKRGRD7LUZ6AP4ARHVFHOCYPJY3NPNA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:50.701331519 (8h5m57.073910864s) [noise=true , address=45.91.116.12:4227 , piece_id=A25PGAY3W24PI2WFW6NYMWVSW3NQK4DC5ZJMC4VISWYQICT3SJQA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:50.701382838 (8h5m57.073962183s) [seg_pos={0 0}, upload=6403]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 06:56:50.701484933 (8h5m57.074064288s) [upload=6403]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 06:56:50.701489862 (8h5m57.074069218s) [upload=6403]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 06:56:51.003338357 (8h5m57.375917732s) [upload=6403]: Done uploading segments... Jul 19 06:56:51.003368886 (8h5m57.375948241s) [upload=6403]: Upload finished. err: <nil> Duration: 5.664299298s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [.X..........................................................] duration: 5.831522043s FAIL (2025-07-19 07:41:53 UTC) orbiter warning: ======ERROR====== uplink: metainfo: metaclient: context deadline exceeded ======LOGS====== ARML7R2U7MK5UBOQLTMNEEL2KMPQ, seg_pos={0 0}, upload=6394]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12mj4gkN1YNsnfjc9svNEhUo8EAgEmHe7yj4kScHCasB9RwVNwZ): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:01.174242029 (8h5m7.546821384s) [noise=true , address=104.36.82.10:4079 , piece_id=CQZTVCO3IEWAGADQCO7PUGO3ZYPMQXKHZ2UPR4X3WXR4UOXD23DQ, seg_pos={0 0}, upload=6394]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XgHpMHksVDkRiDokvccpMMCNXfvH29SBPspkNrLcZuPE2fdud): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:01.174255615 (8h5m7.54683497s) [seg_pos={0 0}, upload=6394]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 06:56:01.174325238 (8h5m7.546904634s) [upload=6394]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 06:56:01.174333464 (8h5m7.546912819s) [upload=6394]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 06:56:01.474962706 (8h5m7.847542061s) [upload=6394]: Done uploading segments... Jul 19 06:56:01.474982223 (8h5m7.847561578s) [upload=6394]: Upload finished. err: <nil> Jul 19 06:56:48.315803186 (8h5m54.688382541s) [upload=6403]: Starting upload Jul 19 06:56:48.315875724 (8h5m54.688455079s) [upload=6403]: Done starting upload Jul 19 06:56:48.315947812 (8h5m54.688527167s) [upload=6403]: Uploading segments... Jul 19 06:56:48.316122416 (8h5m54.688701761s) [upload=6403]: Got next segment. Inline: false Jul 19 06:56:48.316150149 (8h5m54.688729494s) [upload=6403]: Flush batch item: pb.BatchRequestItem_ObjectBegin Jul 19 06:56:48.316155278 (8h5m54.688734633s) [upload=6403]: Flush batch item: pb.BatchRequestItem_SegmentBegin Jul 19 06:56:48.549320531 (8h5m54.921899886s) [seg_pos={0 0}, upload=6403]: Begin upload segment... Jul 19 06:56:48.549361600 (8h5m54.921940955s) [seg_pos={0 0}, upload=6403]: Done begin upload segment. Jul 19 06:56:48.549382059 (8h5m54.921961404s) [noise=true , address=77.90.31.21:4877 , piece_id=EDLQFKGHOTHTLWL7PPUOMRTA6JIL7TF7HU22JNO2EVVXETFLUHYQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549433146 (8h5m54.922012501s) [noise=true , address=104.36.82.13:4185 , piece_id=JHS6G7M7N5IVY2VPSPHJPVRZQHQJIYBU54KKUW47COZP2YGQ2XGA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549694476 (8h5m54.922273831s) [noise=true , address=45.91.116.5:4299 , piece_id=OHEAHX6PK3JW6K3BYMJQYK4HDNP6K7DEJOW7EOQKK2TC4P6NW4CA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549875533 (8h5m54.922454888s) [noise=true , address=45.91.116.13:4153 , piece_id=MPAR7SWMVQ236Z76HU7FC3LMQ4NRUD3TN64HT3MQRWTWGBQ6YDSQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.549937652 (8h5m54.922517007s) [noise=true , address=45.91.116.8:4045 , piece_id=G5QHS6Y4T2JY7PYOXKUGSZM75F4NWF37L45H434N5WOZKEDUMPWA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550155920 (8h5m54.922735275s) [noise=true , address=45.91.116.11:4117 , piece_id=BZQFV37K7LE4Z2K54HPA6PBIVNZEC6SBX4HYUE6YGRCHLYUOOEBA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550269487 (8h5m54.922848832s) [noise=true , address=45.91.116.12:4227 , piece_id=A25PGAY3W24PI2WFW6NYMWVSW3NQK4DC5ZJMC4VISWYQICT3SJQA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550430595 (8h5m54.92300994s) [noise=true , address=104.36.82.15:4259 , piece_id=HAQGLMVTGJXXK334AOVADV45LX4MBWBSDOUHZ7HXYB4CZJ6SWM5A, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550495319 (8h5m54.923074664s) [noise=true , address=77.90.31.19:4841 , piece_id=WJMRBXWLIAPBM4YW6SZDQM3A536PDDRMOJF5R4J2LVRDVJPULP4A, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550699370 (8h5m54.923278715s) [noise=true , address=45.91.116.4:4938 , piece_id=ZIJIIBKIGQY2BRI3DVAEWWLPPPUDB2RYMYV2EAD3EZLO2K2P2WTA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550722122 (8h5m54.923301477s) [noise=true , address=77.90.31.15:4698 , piece_id=6PREKXY6754B6THCWIZRYR3AO3S2XGPR4GSI7WQ2UNJW2NDWAYEQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550951211 (8h5m54.923530566s) [noise=true , address=77.90.31.10:4520 , piece_id=PV2J4IO3YYWYVEKXTS4C2CT3BIFNDJNJOYCYDQC5LQC2B4TZCU3A, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.550955099 (8h5m54.923534444s) [noise=true , address=104.36.82.2:4731 , piece_id=MIOJJFTFCRXMSFE4FU2AKBFTPVIX5JEGOJV5FIX5OGUEUV2OSW7Q, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.551183094 (8h5m54.923762439s) [noise=true , address=77.90.31.12:4589 , piece_id=AMXTBSD6WO4UEXIIXO6L5ZQDD3AYUZ75PIMNMX6CXQTPAUS6MKNQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.551205177 (8h5m54.923784522s) [noise=true , address=104.36.82.14:4222 , piece_id=S2E5GVHNN4AVBWUCQ7ZF4VTBMCE44VKD3ONII57T52EQAGAQBEBQ, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:48.551419828 (8h5m54.923999173s) [noise=true , address=45.91.116.10:4083 , piece_id=4AIGZG7ABSINNQYFQLPC3ZKRGRD7LUZ6AP4ARHVFHOCYPJY3NPNA, seg_pos={0 0}, upload=6403]: Uploading piece... Jul 19 06:56:50.313191049 (8h5m56.685770404s) [upload=6403]: Next returned nil segment Jul 19 06:56:50.313210367 (8h5m56.685789712s) [upload=6403]: Waiting for error group managing segments... Jul 19 06:56:50.602094226 (8h5m56.974673571s) [noise=true , address=45.91.116.4:4938 , piece_id=ZIJIIBKIGQY2BRI3DVAEWWLPPPUDB2RYMYV2EAD3EZLO2K2P2WTA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.606309361 (8h5m56.978888716s) [noise=true , address=104.36.82.14:4222 , piece_id=S2E5GVHNN4AVBWUCQ7ZF4VTBMCE44VKD3ONII57T52EQAGAQBEBQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.610631160 (8h5m56.983210515s) [noise=true , address=45.91.116.5:4299 , piece_id=OHEAHX6PK3JW6K3BYMJQYK4HDNP6K7DEJOW7EOQKK2TC4P6NW4CA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.613288184 (8h5m56.985867539s) [noise=true , address=45.91.116.8:4045 , piece_id=G5QHS6Y4T2JY7PYOXKUGSZM75F4NWF37L45H434N5WOZKEDUMPWA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.624484997 (8h5m56.997064352s) [noise=true , address=45.91.116.11:4117 , piece_id=BZQFV37K7LE4Z2K54HPA6PBIVNZEC6SBX4HYUE6YGRCHLYUOOEBA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.626515311 (8h5m56.999094656s) [noise=true , address=77.90.31.21:4877 , piece_id=EDLQFKGHOTHTLWL7PPUOMRTA6JIL7TF7HU22JNO2EVVXETFLUHYQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.627277630 (8h5m56.999856975s) [noise=true , address=77.90.31.19:4841 , piece_id=WJMRBXWLIAPBM4YW6SZDQM3A536PDDRMOJF5R4J2LVRDVJPULP4A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.642423930 (8h5m57.015003285s) [noise=true , address=77.90.31.15:4698 , piece_id=6PREKXY6754B6THCWIZRYR3AO3S2XGPR4GSI7WQ2UNJW2NDWAYEQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.644061784 (8h5m57.016641129s) [noise=true , address=77.90.31.12:4589 , piece_id=AMXTBSD6WO4UEXIIXO6L5ZQDD3AYUZ75PIMNMX6CXQTPAUS6MKNQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.647067965 (8h5m57.01964732s) [noise=true , address=77.90.31.10:4520 , piece_id=PV2J4IO3YYWYVEKXTS4C2CT3BIFNDJNJOYCYDQC5LQC2B4TZCU3A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.678272179 (8h5m57.050851525s) [noise=true , address=104.36.82.13:4185 , piece_id=JHS6G7M7N5IVY2VPSPHJPVRZQHQJIYBU54KKUW47COZP2YGQ2XGA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.684092075 (8h5m57.05667143s) [noise=true , address=45.91.116.13:4153 , piece_id=MPAR7SWMVQ236Z76HU7FC3LMQ4NRUD3TN64HT3MQRWTWGBQ6YDSQ, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.701086190 (8h5m57.073665535s) [noise=true , address=104.36.82.15:4259 , piece_id=HAQGLMVTGJXXK334AOVADV45LX4MBWBSDOUHZ7HXYB4CZJ6SWM5A, seg_pos={0 0}, upload=6403]: Done uploading piece. err: <nil> Jul 19 06:56:50.701112961 (8h5m57.073692306s) [seg_pos={0 0}, upload=6403]: Segment reached optimal threshold of 13 pieces. Jul 19 06:56:50.701241187 (8h5m57.073820542s) [noise=true , address=104.36.82.2:4731 , piece_id=MIOJJFTFCRXMSFE4FU2AKBFTPVIX5JEGOJV5FIX5OGUEUV2OSW7Q, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1m6CiDtKdTVJSdeLWBbGZh6LS66pnTP8YoVwFpKSccTKoJfok9): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:50.701311190 (8h5m57.073890545s) [noise=true , address=45.91.116.10:4083 , piece_id=4AIGZG7ABSINNQYFQLPC3ZKRGRD7LUZ6AP4ARHVFHOCYPJY3NPNA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:123WHuy89PAwkQ6AhXenBsQM6HdptYmwJWWqNkGC7P1boNd4K8B): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:50.701331519 (8h5m57.073910864s) [noise=true , address=45.91.116.12:4227 , piece_id=A25PGAY3W24PI2WFW6NYMWVSW3NQK4DC5ZJMC4VISWYQICT3SJQA, seg_pos={0 0}, upload=6403]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12ETZh5HbRYk386nSYJSd4DvQ2XbMw5XBnyyPMfsxwHJaSHaQEH): protocol: expected piece hash; context canceled; EOF Jul 19 06:56:50.701382838 (8h5m57.073962183s) [seg_pos={0 0}, upload=6403]: Done waiting for segment. successful: 13 optimal: 13 errs: <nil> Jul 19 06:56:50.701484933 (8h5m57.074064288s) [upload=6403]: Flush batch item: pb.BatchRequestItem_SegmentCommit Jul 19 06:56:50.701489862 (8h5m57.074069218s) [upload=6403]: Flush batch item: pb.BatchRequestItem_ObjectCommit Jul 19 06:56:51.003338357 (8h5m57.375917732s) [upload=6403]: Done uploading segments... Jul 19 06:56:51.003368886 (8h5m57.375948241s) [upload=6403]: Upload finished. err: <nil> Duration: 5.831545799s core uplink us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)