Orbiter History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [......X...X.................X.............X.....X...........] duration: 5.84921054s FAIL (2025-08-17 05:13:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.849238784s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [........X.....X.....................XX......................] duration: 1.883519084s FAIL (2025-08-17 05:13:47 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.883538249s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 5 out of 60 is failed (threshold=3) [.......X...X.................X.............X.....X..........] duration: 5.607947065s FAIL (2025-08-17 05:13:28 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.607965902s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [X.......X...X.................X.............X.....X.........] duration: 5.63831045s FAIL (2025-08-17 05:12:48 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.638346368s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.........X.....X.....................XX.....................] duration: 1.815810503s FAIL (2025-08-17 05:12:48 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.815843507s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..........X.....X.....................XX....................] duration: 1.798900393s FAIL (2025-08-17 05:12:28 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.798920922s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 6 out of 60 is failed (threshold=3) [.X.......X...X.................X.............X.....X........] duration: 5.666031328s FAIL (2025-08-17 05:12:08 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.666060374s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...........X.....X.....................XX...................] duration: 2.111357589s FAIL (2025-08-17 05:11:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 2.111388237s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [X.X.......X...X.................X.............X.....X.......] duration: 6.007233101s FAIL (2025-08-17 05:10:58 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 6.007307723s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [............X.....X.....................XX..................] duration: 1.849089952s FAIL (2025-08-17 05:10:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.84910989s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 7 out of 60 is failed (threshold=3) [.X.X.......X...X.................X.............X.....X......] duration: 5.595858536s FAIL (2025-08-17 05:09:44 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.595884387s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.............X.....X.....................XX.................] duration: 2.146903582s FAIL (2025-08-17 05:09:33 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 2.146929881s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..............X.....X.....................XX................] duration: 1.846007243s FAIL (2025-08-17 05:08:42 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.846033945s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.97719605s FAIL (2025-08-17 05:08:39 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.977216419s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 8 out of 60 is failed (threshold=3) [X.X.X.......X...X.................X.............X.....X.....] duration: 5.515455519s FAIL (2025-08-17 05:08:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.515477632s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 9 out of 60 is failed (threshold=3) [XX.X.X.......X...X.................X.............X.....X....] duration: 5.671171552s FAIL (2025-08-17 05:07:56 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.67119603s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.485245049s FAIL (2025-08-17 05:07:55 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.485274977s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...............X.....X.....................XX...............] duration: 2.150285358s FAIL (2025-08-17 05:07:36 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 2.150317107s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [XXX.X.X.......X...X.................X.............X.....X...] duration: 5.742766611s FAIL (2025-08-17 05:07:16 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.742792721s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.XX.........................................................] duration: 6.460969788s FAIL (2025-08-17 05:07:16 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.46100727s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [................X.....X.....................XX..............] duration: 2.082735921s FAIL (2025-08-17 05:07:06 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 2.08275599s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.................X.....X.....................XX.............] duration: 1.84738963s FAIL (2025-08-17 05:06:50 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.847421251s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [..XX........................................................] duration: 6.799201s FAIL (2025-08-17 05:06:23 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.79954101s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.06548049s FAIL (2025-08-17 05:06:22 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.065506501s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [XXXX.X.X.......X...X.................X.............X.....X..] duration: 5.818485733s FAIL (2025-08-17 05:06:11 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.818511521s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.058020579s FAIL (2025-08-17 05:05:51 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.058052581s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [...XX.......................................................] duration: 6.904142365s FAIL (2025-08-17 05:05:43 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.904208321s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [..................X.....X.....................XX............] duration: 2.014953636s FAIL (2025-08-17 05:05:40 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 2.014980407s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [.XXXX.X.X.......X...X.................X.............X.....X.] duration: 5.928935287s FAIL (2025-08-17 05:05:36 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 5.928953472s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.170390338s FAIL (2025-08-17 05:05:28 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.170413252s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [...................X.....X.....................XX...........] duration: 2.216790562s FAIL (2025-08-17 05:05:21 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 2.216813805s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 1 out of 60 is failed (threshold=3) [X...........................................................] duration: 6.325750162s FAIL (2025-08-17 05:05:13 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.325772986s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 5.798324399s FAIL (2025-08-17 05:05:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.798353454s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 5.763324868s FAIL (2025-08-17 05:05:10 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.76335124s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [....................X.....X.....................XX..........] duration: 1.818133961s FAIL (2025-08-17 05:05:00 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.818161864s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [....XX......................................................] duration: 6.894638861s FAIL (2025-08-17 05:04:59 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.894656496s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 2 out of 60 is failed (threshold=3) [XX..........................................................] duration: 6.825297976s FAIL (2025-08-17 05:04:53 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.825317764s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 11 out of 60 is failed (threshold=3) [..XXXX.X.X.......X...X.................X.............X.....X] duration: 4.213050118s FAIL (2025-08-17 05:04:52 UTC) orbiter warning: operation: ListObjects ======ERROR====== unable to find key in bucket: sj://uptime-checks-placement-20/orbiter_us_0/4M ======LOGS====== Aug 17 05:04:48.666350748 (203h38m43.097430869s) [upload=211828]: Starting upload Aug 17 05:04:48.666396716 (203h38m43.097476847s) [upload=211828]: Done starting upload Aug 17 05:04:48.666479835 (203h38m43.097559956s) [upload=211828]: Uploading segments... Aug 17 05:04:48.666607570 (203h38m43.097687691s) [upload=211828]: Got next segment. Inline: false Aug 17 05:04:48.666638419 (203h38m43.09771855s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:04:48.666642697 (203h38m43.097722828s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:04:49.164031930 (203h38m43.595112061s) [seg_pos={0 0}, upload=211828]: Begin upload segment... Aug 17 05:04:49.164072468 (203h38m43.595152589s) [seg_pos={0 0}, upload=211828]: Done begin upload segment. Aug 17 05:04:49.164093037 (203h38m43.595173168s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164302167 (203h38m43.595382298s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164522900 (203h38m43.595603031s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164742139 (203h38m43.59582227s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.164817093 (203h38m43.595897224s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165028788 (203h38m43.596108909s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165131053 (203h38m43.596211174s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165306680 (203h38m43.596386811s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165536098 (203h38m43.596616229s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165595452 (203h38m43.596675583s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165819340 (203h38m43.596899471s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.165880648 (203h38m43.596960779s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166037537 (203h38m43.597117668s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166128602 (203h38m43.597208733s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166256356 (203h38m43.597336487s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166366357 (203h38m43.597446488s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166477189 (203h38m43.59755732s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:49.166696639 (203h38m43.59777677s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Uploading piece... Aug 17 05:04:50.286395117 (203h38m44.717475249s) [upload=211828]: Next returned nil segment Aug 17 05:04:50.286409705 (203h38m44.717489836s) [upload=211828]: Waiting for error group managing segments... Aug 17 05:04:50.572897900 (203h38m45.003978032s) [noise=true , address=66.23.193.175:4561 , piece_id=KZTGKBLBBHZZWQPHLBI4Y6GK2PFOMY4Q3BNEJE3WS2TBFB3PPTSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.049689573 (203h38m45.480769694s) [noise=true , address=45.91.116.5:4305 , piece_id=IVCX4RWD775ZZSCR27ZYYQRXW36446AKXSIGNXAKF6YJYK6A6VQA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.054105883 (203h38m45.485186004s) [noise=true , address=45.91.116.7:4041 , piece_id=DK3HUF4WMCKE6VAWKQ5KDPL3SM7PADSSJU6ZJL2XGE2FZ2S74LRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.055297733 (203h38m45.486377864s) [noise=true , address=77.90.31.16:4761 , piece_id=O474HOHDL45NC4VKKMYBCNPLGYETNARXFWL45OOGBN7YSX24FZSA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.062828492 (203h38m45.493908613s) [noise=true , address=104.36.82.18:4400 , piece_id=RKBACKXSYZ3YI5A4MXGFLNNVDWYTKD4OYFZKOKUFSFUJQK6G2NUQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.067281833 (203h38m45.498361964s) [noise=true , address=104.36.82.37:4659 , piece_id=EFPSHVAK5G3X2C5QHVAVOKGKWKN4MIMHFZIYA65FUYRCUQKOV3QA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.070340486 (203h38m45.501420607s) [noise=true , address=104.36.82.22:4514 , piece_id=LDOEBRZEIX5ZHZG7CC66A5R6NITKVOYTADLEQUHJHGIHBLPDZYRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.117817658 (203h38m45.548897789s) [noise=true , address=77.90.31.13:4658 , piece_id=CSCSBLQWAEL3766AYNSZ3ILQ2GMSB2KODPHQRAGJW5TGMLPTCHKQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.125475180 (203h38m45.556555311s) [noise=true , address=104.36.82.60:4344 , piece_id=SMC7E62A35KNJNU6HGZ2FWTCA6V23ZHW6HNPUC4SOTXNSHYXAHCQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.132901780 (203h38m45.563981911s) [noise=true , address=79.127.201.215:4354 , piece_id=4U5QQCPZP374UCOUJEKZGMOFKEERQOMBFZIJKC7CVEJWZBVWCLRQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.152343280 (203h38m45.583423411s) [noise=true , address=104.36.82.36:4619 , piece_id=YFYB3VICLKDWCQSVZDASYNTB6MM55WG35JVZYE23VLXBFMQIDFCA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.160602643 (203h38m45.591682774s) [noise=true , address=104.36.82.23:4477 , piece_id=XRUN2OCCJEQ3M27JWHJXIX7C6SCUZFW2GBJ5Y6M5HYDE77ONEB6A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.185622466 (203h38m45.616702597s) [noise=true , address=104.36.82.44:4889 , piece_id=5CNDRXZLODI3HLVK6DLRMF6ISXGGVDTKQW5SUR2KD2DS66DNB55Q, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196946053 (203h38m45.628026184s) [noise=true , address=77.90.31.15:4712 , piece_id=PSRP7HL7WZKAG5NSJGSTMFAUVHB665NDWQMQ3U2W33XJIX55ZDSQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: <nil> Aug 17 05:04:51.196971011 (203h38m45.628051142s) [seg_pos={0 0}, upload=211828]: Segment reached optimal threshold of 14 pieces. Aug 17 05:04:51.197081632 (203h38m45.628161763s) [noise=true , address=104.36.82.2:4750 , piece_id=JNG4VACRCG52CKSJLHXAP2ASMKR6DG3AOLPPZ5YPZDHOBI3ETMYQ, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12XoDDL5HwunayYqgQDzvxMsgFnTx1kUU6aiyvhsRLucnR9eTUz): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197201852 (203h38m45.628281983s) [noise=true , address=104.36.82.40:4953 , piece_id=GCNSNWTA2ZYFTAHU7PHJE65Y5RDF2XWLPVYRGK4P3JAHXRTUQM3A, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12dhNW98YkWKWzXBnwmTCvzaw9CtvxRW8ELJnAjHnATFhpM1oKn): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197226549 (203h38m45.62830668s) [noise=true , address=104.36.82.5:4850 , piece_id=H2JYSYRARRPRBSY3JUMZGFVMIWITH6YWQ4LKTSD5L5C7H7Q4OSRA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1obTuzA1eVY69zki5vcH4cUjgZJzXnbKaj17TLjCLhDhLi7ZEi): protocol: expected piece hash; context canceled; EOF Aug 17 05:04:51.197291543 (203h38m45.628371674s) [noise=true , address=45.91.116.11:4127 , piece_id=2WU66KSBO74VIH67SWSI75HKKCS72UVXK5SEYEA3JNINGR3ZCETA, seg_pos={0 0}, upload=211828]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:1MuGqtY6SnQrWgxJKjb15C6eLdKPbNijPHhH6D8sfrvhb6iKWG): protocol: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection; piecestore: piecestore close: write tcp 188.166.225.170:44390->45.91.116.11:4127: use of closed network connection Aug 17 05:04:51.197309879 (203h38m45.62839s) [seg_pos={0 0}, upload=211828]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:04:51.197372989 (203h38m45.62845312s) [upload=211828]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:04:51.197379733 (203h38m45.628459904s) [upload=211828]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:04:51.772823365 (203h38m46.203903496s) [upload=211828]: Done uploading segments... Aug 17 05:04:51.772843172 (203h38m46.203923303s) [upload=211828]: Upload finished. err: <nil> Duration: 4.213075157s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 5.773744075s FAIL (2025-08-17 05:04:35 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 5.773806143s core uplink us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [.XXX........................................................] duration: 5.836722138s FAIL (2025-08-17 05:04:25 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.836746625s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 6.07481933s FAIL (2025-08-17 05:04:14 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 13 but got 10) ======LOGS====== Aug 17 04:23:03.873697274 (202h56m58.304777405s) [upload=211112]: Starting upload Aug 17 04:23:03.873756368 (202h56m58.304836499s) [upload=211112]: Done starting upload Aug 17 04:23:03.873839848 (202h56m58.304919989s) [upload=211112]: Uploading segments... Aug 17 04:23:03.873950850 (202h56m58.305030981s) [upload=211112]: Got next segment. Inline: false Aug 17 04:23:03.873976509 (202h56m58.30505664s) [upload=211112]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:23:03.873980757 (202h56m58.305060888s) [upload=211112]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 04:23:04.113727713 (202h56m58.544807844s) [seg_pos={0 0}, upload=211112]: Begin upload segment... Aug 17 04:23:04.113774051 (202h56m58.544854182s) [seg_pos={0 0}, upload=211112]: Done begin upload segment. Aug 17 04:23:04.113820890 (202h56m58.544901021s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.113845989 (202h56m58.54492612s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114195037 (202h56m58.545275168s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114281412 (202h56m58.545361543s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114482115 (202h56m58.545562237s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114570385 (202h56m58.545650516s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114757392 (202h56m58.545837523s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.114874727 (202h56m58.545954858s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115030715 (202h56m58.546110846s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115194288 (202h56m58.546274419s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115307967 (202h56m58.546388098s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115535692 (202h56m58.546615823s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.115754130 (202h56m58.546834261s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116079182 (202h56m58.547159323s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116350181 (202h56m58.547430312s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:04.116554471 (202h56m58.547634602s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Uploading piece... Aug 17 04:23:05.868765063 (202h57m0.299845195s) [upload=211112]: Next returned nil segment Aug 17 04:23:05.868801043 (202h57m0.299881164s) [upload=211112]: Waiting for error group managing segments... Aug 17 04:23:06.147430529 (202h57m0.57851066s) [noise=true , address=45.91.116.12:4227 , piece_id=XZR3AWJG57RUB4TN7SUWNIWWD2XGKCMDXSWSM74WS3ATHS32EBJQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.182400314 (202h57m0.613480445s) [noise=true , address=45.91.116.12:4225 , piece_id=EUPPM5NRIU5MCZBER3VGZBZEMZZ65Q2MOY2DXKFBIPFJ53M5TJHQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.191910532 (202h57m0.622990663s) [noise=true , address=45.91.116.4:4938 , piece_id=X5MDNBZYRYYOXDEAMGIOD5CXX7PGAMTFRMROOVTR6NIAYQ5VMXGA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.197391631 (202h57m0.628471782s) [noise=true , address=45.91.116.9:4261 , piece_id=PNOEBFIPLWM4B2QLGJUVEBTT23J5AVRFTWTTKLHFUJ5MNJTSM2JA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.198069487 (202h57m0.629149618s) [noise=true , address=45.91.116.9:4263 , piece_id=VT734KJZ7T3CZ3W3P4M4ZCYKOLLTPP3ILFRWL26Z7K4LWBDZV4FA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.200597014 (202h57m0.631677145s) [noise=true , address=45.91.116.7:4009 , piece_id=IVTDHCTNJEEO3QATGN25YH5Y2MSEQUWJIFPXSJWINUEXDHQAGELA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.288053251 (202h57m0.719133372s) [noise=true , address=45.91.116.7:4010 , piece_id=HNJU4MPWT67V26J5XDCTHFEOOJL2HS3VAOHB4EIPQVF4D2NNRFMA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.298810435 (202h57m0.729890566s) [noise=true , address=45.91.116.14:4189 , piece_id=2P3GOZDEMJGBAK2F6SY7NJNQUTS5ZYM4B4IMQFAA5TYFARD4DVCQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.301049520 (202h57m0.732129651s) [noise=true , address=45.91.116.13:4155 , piece_id=CO3R5XZGH3NJADAFDILPNZEKKFJSRPB4JOFBBCSYYQAD3KNRABNA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:06.402776985 (202h57m0.833857116s) [noise=true , address=77.90.31.16:4735 , piece_id=G5YGF7LXNPBU2TDRYPAFPVYQSTWUGG7Y6YM2Q6RWMCZY5RVNM4BA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: <nil> Aug 17 04:23:33.223003412 (202h57m27.654083543s) [noise=true , address=104.36.82.5:4838 , piece_id=IYA7ERDAAFYUGC4666CBEX5HZEXZVFPES6JUYRZROY25OAPGVEQQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1iSxSmovuEAgNmjE3yCqXKtiRijYbcEcTtUdNxU3iC1ozTxfwL, address:104.36.82.5:4838): protocol: write tcp 188.166.225.170:42258->104.36.82.5:4838: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223050702 (202h57m27.654130823s) [noise=true , address=104.36.82.8:4969 , piece_id=3D3YAZSP7CD6IFZGDKPNKIJPWZ2OJFFVJCSMTO42VNJEFVU3ORHA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1HxTyEaQTErcZJWMYLsND9Xa8Nwaa5e1f4x2bs7UQzSRiW7buX, address:104.36.82.8:4969): protocol: write tcp 188.166.225.170:52308->104.36.82.8:4969: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223090619 (202h57m27.65417075s) [noise=true , address=104.36.82.15:4257 , piece_id=Q5GLP6TS7N7CM4DO3LWDCDR6O76OFZ6IVQJC5VQWMA5XLNWZ2E4A, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12shbJLfcGs27jH9tFC4L5bQCSNnkatQSXEUyVvDBCJ6oPLLeMK, address:104.36.82.15:4257): protocol: write tcp 188.166.225.170:53078->104.36.82.15:4257: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223104976 (202h57m27.654185097s) [noise=true , address=104.36.82.4:4804 , piece_id=MSAHN2DERVG2HL7KZMKIWYUQYO6LRIXGUNWJFXJDDGOWZT54FPVA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12M1GKTnn8Uex751kqW5NhedsDiRb7GR7u1Dd9zwDm6XMzcRL2g, address:104.36.82.4:4804): protocol: write tcp 188.166.225.170:56176->104.36.82.4:4804: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223149581 (202h57m27.654229712s) [noise=true , address=104.36.82.12:4151 , piece_id=2GGYMOWZ2FTWDNWNQRKSAH7UVJ4R6MLRGC6OHNVV3KTVIIMO4QIQ, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:1w6pwZMdeQKjt92qx6WbTWU5VPYJto5gS87f6iKvCBZH6LfyeB, address:104.36.82.12:4151): protocol: write tcp 188.166.225.170:40582->104.36.82.12:4151: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223157536 (202h57m27.654237667s) [noise=true , address=104.36.82.9:4005 , piece_id=FAR4B5KAALBOACKANEFKGECI2ZVXSHBNYDEIWVGDU2RNBNQXLMXA, seg_pos={0 0}, upload=211112]: Done uploading piece. err: ecclient: upload failed (node:12YnzPGyppEZ97NvHxEQiSK18T1bctGqPzdDrXzLGeCSRUBj2WJ, address:104.36.82.9:4005): protocol: write tcp 188.166.225.170:32954->104.36.82.9:4005: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:33.223177415 (202h57m27.654257536s) [seg_pos={0 0}, upload=211112]: Done waiting for segment. successful: 10 optimal: 13 errs: <nil> Aug 17 04:23:33.448862924 (202h57m27.879943045s) [upload=211112]: Done uploading segments... Aug 17 04:23:33.448886830 (202h57m27.879966961s) [upload=211112]: Upload finished. err: failed to upload enough pieces (needed at least 13 but got 10) Duration: 6.074842824s core uplink us-select-1
checkers.Gateway{"https://s3.de2eae4e.storjapi.io/", AuthURL: "https://auth.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us1.placement_20) antiblip: 2 out of 60 is failed (threshold=3) [.....XX.....................................................] duration: 6.684167704s FAIL (2025-08-17 05:04:10 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.6847501s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [..XXX.......................................................] duration: 5.529041499s FAIL (2025-08-17 05:04:09 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 5.529069282s core uplink us-select-1 us1-placement-20
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 3 out of 60 is failed (threshold=3) [XXX.........................................................] duration: 6.545822316s FAIL (2025-08-17 05:04:06 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.545842265s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote) antiblip: 10 out of 60 is failed (threshold=3) [...XXXX.X.X.......X...X.................X.............X.....] duration: 5.546974696s FAIL (2025-08-17 05:03:55 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/4M") ======LOGS====== Aug 17 05:00:40.741189040 (203h34m35.172269171s) [upload=211759]: Starting upload Aug 17 05:00:40.741248243 (203h34m35.172328374s) [upload=211759]: Done starting upload Aug 17 05:00:40.741340399 (203h34m35.17242052s) [upload=211759]: Uploading segments... Aug 17 05:00:40.741554589 (203h34m35.17263472s) [upload=211759]: Got next segment. Inline: false Aug 17 05:00:40.741584907 (203h34m35.172665038s) [upload=211759]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 05:00:40.741605587 (203h34m35.172685708s) [upload=211759]: Flush batch item: pb.BatchRequestItem_SegmentBegin Aug 17 05:00:41.328767220 (203h34m35.759847361s) [seg_pos={0 0}, upload=211759]: Begin upload segment... Aug 17 05:00:41.328863595 (203h34m35.759943736s) [seg_pos={0 0}, upload=211759]: Done begin upload segment. Aug 17 05:00:41.329892563 (203h34m35.760972704s) [noise=true , address=104.36.82.36:4621 , piece_id=HSY6SEM2ZKEGPA2KK2IDM5RDDHKTVLJ5LQBOVNNVOCEN54LR23TQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.329906781 (203h34m35.760986902s) [noise=true , address=66.23.193.175:4563 , piece_id=TMEUYYYD34T7N3N7XA6STWBUGVIQDSXHRPVKHQK4MZEUGNHX4S5Q, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.330378463 (203h34m35.761458594s) [noise=true , address=77.90.31.10:4535 , piece_id=TMDTBWNQ37XOXS6VBVIIISIXU6NLKGL2VHGFRWXBEEAE3P2MRNHA, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.331010312 (203h34m35.762090443s) [noise=true , address=104.36.82.39:4748 , piece_id=ABHZEFBQSOTUKYLERY46TNQAACQA7IP5QUGZRCAQEPW6N6EJHUJQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.331172532 (203h34m35.762252653s) [noise=true , address=77.90.31.16:4761 , piece_id=3YH6JXG2IG4P2DO5QHQGQWFREGG7ML3CKLEW7L7L3XHZ7ALYQNMQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.331494569 (203h34m35.7625747s) [noise=true , address=104.36.82.17:4329 , piece_id=FXUITOWR3VKOYQK7REQWBUTFV6T3BIOQMGAG6XJDFNHOGPJ7XB2Q, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.331816645 (203h34m35.762896776s) [noise=true , address=156.146.43.225:4453 , piece_id=6JZUO2KWLPM4F5NXHGXJQ7OSPC535TXQ7K3YOZAGOKWQNJDFBZ7A, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.331937987 (203h34m35.763018118s) [noise=true , address=104.36.82.5:4852 , piece_id=LG5ELOKLASDVYG5SJGXNJ4GKHMCUVCTPF26UFYADDOR7PNZSYE4A, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.332421693 (203h34m35.763501824s) [noise=true , address=77.90.31.12:4610 , piece_id=V5XT6PATLMIEOTQR6GZUJH4ASGOGBFY7J3TTVMDZNNJCXLROVOMA, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.332532525 (203h34m35.763612656s) [noise=true , address=104.36.82.59:4331 , piece_id=3XTOPCDXYEHGKSONB4SOMCIYEATQTQ6UJSO35VYJ3EIKF5GPC7OQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.332886522 (203h34m35.763966653s) [noise=true , address=104.36.82.44:4870 , piece_id=NKEPA3RLLUVK4RF4XRCY5RTSCTUVZL2GTRSFTVMRZIJ57IAAKPSQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.333000080 (203h34m35.764080211s) [noise=true , address=79.127.201.215:4359 , piece_id=2LTZPT7ROWAYS4FBZSPCB5A5NGMTJ2BT2K6QBDLE7S47GELVPK4Q, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.333394845 (203h34m35.764474976s) [noise=true , address=156.146.43.225:4459 , piece_id=OLWWYL72N4DBJIE3WWI3YBB3S36OK4PZECY35YSTRZFOM3JGWSQQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.333738292 (203h34m35.764818423s) [noise=true , address=77.90.31.11:4575 , piece_id=QEC474LJ5VRTPR2TAYNB7XPREJPJTQHJ2SKLIIAT7IU6SHSCTU5Q, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.334205716 (203h34m35.765285848s) [noise=true , address=104.36.82.23:4450 , piece_id=XGWFSLDSS336YVE43M5GLKUMLBTGKDSLIADGQFUX6W7LDZIC5AYQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.334444332 (203h34m35.765524463s) [noise=true , address=104.36.82.60:4344 , piece_id=BUNMEHGSM64M5CTPVZ4KKXMOEFCE4KWHHSTV37ZSDILNIQK5RVQQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.334742473 (203h34m35.765822604s) [noise=true , address=77.90.31.14:4678 , piece_id=YUBUDJTG5MQKO47QMITLYGUZ6VE7LADJDE7QY2CE2DAZKJLK7NKQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:41.334827285 (203h34m35.765907416s) [noise=true , address=79.127.201.214:4316 , piece_id=VX3I4PMTWFOMJ62KE6LW6HWBGIMAFZCYH3ZBTT5V5GZ56IGJ6BEQ, seg_pos={0 0}, upload=211759]: Uploading piece... Aug 17 05:00:42.493214350 (203h34m36.924294481s) [upload=211759]: Next returned nil segment Aug 17 05:00:42.493234729 (203h34m36.92431486s) [upload=211759]: Waiting for error group managing segments... Aug 17 05:00:42.787324538 (203h34m37.218404659s) [noise=true , address=66.23.193.175:4563 , piece_id=TMEUYYYD34T7N3N7XA6STWBUGVIQDSXHRPVKHQK4MZEUGNHX4S5Q, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.209310205 (203h34m37.640390337s) [noise=true , address=77.90.31.10:4535 , piece_id=TMDTBWNQ37XOXS6VBVIIISIXU6NLKGL2VHGFRWXBEEAE3P2MRNHA, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.272654437 (203h34m37.703734608s) [noise=true , address=77.90.31.11:4575 , piece_id=QEC474LJ5VRTPR2TAYNB7XPREJPJTQHJ2SKLIIAT7IU6SHSCTU5Q, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.278562522 (203h34m37.709642653s) [noise=true , address=77.90.31.14:4678 , piece_id=YUBUDJTG5MQKO47QMITLYGUZ6VE7LADJDE7QY2CE2DAZKJLK7NKQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.279090813 (203h34m37.710170944s) [noise=true , address=156.146.43.225:4453 , piece_id=6JZUO2KWLPM4F5NXHGXJQ7OSPC535TXQ7K3YOZAGOKWQNJDFBZ7A, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.283605301 (203h34m37.714685432s) [noise=true , address=156.146.43.225:4459 , piece_id=OLWWYL72N4DBJIE3WWI3YBB3S36OK4PZECY35YSTRZFOM3JGWSQQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.286952876 (203h34m37.718033007s) [noise=true , address=104.36.82.36:4621 , piece_id=HSY6SEM2ZKEGPA2KK2IDM5RDDHKTVLJ5LQBOVNNVOCEN54LR23TQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.297235291 (203h34m37.728315422s) [noise=true , address=79.127.201.215:4359 , piece_id=2LTZPT7ROWAYS4FBZSPCB5A5NGMTJ2BT2K6QBDLE7S47GELVPK4Q, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.300883401 (203h34m37.731963532s) [noise=true , address=104.36.82.59:4331 , piece_id=3XTOPCDXYEHGKSONB4SOMCIYEATQTQ6UJSO35VYJ3EIKF5GPC7OQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.322414258 (203h34m37.753494389s) [noise=true , address=104.36.82.23:4450 , piece_id=XGWFSLDSS336YVE43M5GLKUMLBTGKDSLIADGQFUX6W7LDZIC5AYQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.323500056 (203h34m37.754580197s) [noise=true , address=77.90.31.16:4761 , piece_id=3YH6JXG2IG4P2DO5QHQGQWFREGG7ML3CKLEW7L7L3XHZ7ALYQNMQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.326980736 (203h34m37.758060877s) [noise=true , address=77.90.31.12:4610 , piece_id=V5XT6PATLMIEOTQR6GZUJH4ASGOGBFY7J3TTVMDZNNJCXLROVOMA, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.345464623 (203h34m37.776544754s) [noise=true , address=104.36.82.39:4748 , piece_id=ABHZEFBQSOTUKYLERY46TNQAACQA7IP5QUGZRCAQEPW6N6EJHUJQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.353407250 (203h34m37.784487381s) [noise=true , address=104.36.82.17:4329 , piece_id=FXUITOWR3VKOYQK7REQWBUTFV6T3BIOQMGAG6XJDFNHOGPJ7XB2Q, seg_pos={0 0}, upload=211759]: Done uploading piece. err: <nil> Aug 17 05:00:43.353418040 (203h34m37.784498161s) [seg_pos={0 0}, upload=211759]: Segment reached optimal threshold of 14 pieces. Aug 17 05:00:43.353530536 (203h34m37.784610667s) [noise=true , address=79.127.201.214:4316 , piece_id=VX3I4PMTWFOMJ62KE6LW6HWBGIMAFZCYH3ZBTT5V5GZ56IGJ6BEQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:121KF4JuWoJTPMdHSxKwzdAegRDU4FLTJZmDgXLYECv3AiSva6P): protocol: expected piece hash; context canceled; EOF Aug 17 05:00:43.353556284 (203h34m37.784636415s) [noise=true , address=104.36.82.44:4870 , piece_id=NKEPA3RLLUVK4RF4XRCY5RTSCTUVZL2GTRSFTVMRZIJ57IAAKPSQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sDH7ccMQhjePqpbTkkxMAtumNLxpMHiT56cnfaYrUTFYxuPVt): protocol: expected piece hash; context canceled; EOF Aug 17 05:00:43.353590050 (203h34m37.784670181s) [noise=true , address=104.36.82.60:4344 , piece_id=BUNMEHGSM64M5CTPVZ4KKXMOEFCE4KWHHSTV37ZSDILNIQK5RVQQ, seg_pos={0 0}, upload=211759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:127WWD5hr1ZBkRLCHz3jb7kpqqnxPS6VuZ1MyQDmitFbhTSbJNK): protocol: expected piece hash; context canceled; EOF Aug 17 05:00:43.353615659 (203h34m37.78469579s) [noise=true , address=104.36.82.5:4852 , piece_id=LG5ELOKLASDVYG5SJGXNJ4GKHMCUVCTPF26UFYADDOR7PNZSYE4A, seg_pos={0 0}, upload=211759]: Done uploading piece. err: context canceled; ecclient: upload cut due to slow connection (node:12sQvAZS92R7TTJUFjMRieYT6BP4vp2UAan4PHrhPt6m3eHPukN): protocol: expected piece hash; context canceled; EOF Aug 17 05:00:43.353625417 (203h34m37.784705548s) [seg_pos={0 0}, upload=211759]: Done waiting for segment. successful: 14 optimal: 14 errs: <nil> Aug 17 05:00:43.353679591 (203h34m37.784759722s) [upload=211759]: Flush batch item: pb.BatchRequestItem_SegmentCommit Aug 17 05:00:43.353683959 (203h34m37.78476409s) [upload=211759]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 05:00:43.759222005 (203h34m38.190302126s) [upload=211759]: Done uploading segments... Aug 17 05:00:43.759242795 (203h34m38.190322926s) [upload=211759]: Upload finished. err: <nil> Duration: 5.546996017s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote) antiblip: 3 out of 60 is failed (threshold=3) [...XXX......................................................] duration: 6.030629411s FAIL (2025-08-17 05:03:53 UTC) orbiter warning: operation: Commit ======ERROR====== uplink: failed to upload enough pieces (needed at least 14 but got 10) ======LOGS====== g_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761278730 (202h56m21.192358861s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761661883 (202h56m21.192742014s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761716548 (202h56m21.192796679s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.761972016 (202h56m21.193052147s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762254878 (202h56m21.193335009s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762523682 (202h56m21.193603803s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.762864294 (202h56m21.193944415s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763179537 (202h56m21.194259668s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763478499 (202h56m21.19455864s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763757683 (202h56m21.194837814s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.763988195 (202h56m21.195068326s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764273320 (202h56m21.195353451s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764555100 (202h56m21.195635221s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.764757416 (202h56m21.195837547s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.765868993 (202h56m21.196949124s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.767152809 (202h56m21.19823294s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:26.768577886 (202h56m21.199658017s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Uploading piece... Aug 17 04:22:27.934309011 (202h56m22.365389142s) [upload=211101]: Next returned nil segment Aug 17 04:22:27.934342545 (202h56m22.365422666s) [upload=211101]: Waiting for error group managing segments... Aug 17 04:22:28.231985083 (202h56m22.663065214s) [noise=true , address=66.23.193.175:4559 , piece_id=46X42PJIPA6DVC7BG7WPKMPMC44ZBUROEAB5C22M5INXKBJC4KVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.670701256 (202h56m23.101781427s) [noise=true , address=45.91.116.10:4095 , piece_id=JQ2UARQSNKBGHHHBJ6PR45ETR4EYMCXPDEQHEGWFYH52LRJC57UQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.671952431 (202h56m23.103032562s) [noise=true , address=45.91.116.5:4308 , piece_id=F52M3VMHRQWR5LIEEPMVKVBFMOVG76UNNGQTFKSF6ZBD7VEOFWVA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.698468278 (202h56m23.129548409s) [noise=true , address=45.91.116.8:4067 , piece_id=EZAI7ZDUBOOWXYJNTB7ZPOUSBUUG2FKV77QVUOZMNM6TUJSKGYWA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.720171187 (202h56m23.151251318s) [noise=true , address=45.91.116.13:4181 , piece_id=7VN6RQGOTBHQO6HXM5IFTU32PWNVPH6BJZMQNETJEXOYPR2Y7OCQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.731591429 (202h56m23.16267156s) [noise=true , address=45.91.116.14:4203 , piece_id=5LXGQ4JZD2KLW3OMWY5QYRE4MKQNKKSAIMESODKDDT2W33XD45DQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.798148986 (202h56m23.229229117s) [noise=true , address=77.90.31.18:4815 , piece_id=G6PINAFFA6DZRRMKU2XK2LYFOES5WG7PTJ5L2ZPUQHMA4JNSRZRQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.804849037 (202h56m23.235929168s) [noise=true , address=79.127.201.215:4345 , piece_id=JMSKNIZ7QPGBXIEP5RA3S74XAU5NJ2MUUXSFFIAK377YFGO646YQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.833066822 (202h56m23.264147013s) [noise=true , address=77.90.31.15:4732 , piece_id=4AU7UGKXN4TIPSIRR7IMXSF6WHELO6UTOX35N5X7JHOQQXWWJF2A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:22:28.846684710 (202h56m23.277764831s) [noise=true , address=77.90.31.11:4562 , piece_id=PUL6TAVIIU7732IOASNH5EHQMNCK7DWHJMUJVXRNK74T4GUQDGTA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: <nil> Aug 17 04:23:25.582810947 (202h57m20.013891078s) [noise=true , address=104.36.82.34:4558 , piece_id=XGFNEBBVZX5UWJN7DALGM2EDG26POCTSBBAUU7DLYQKS2DNSBNXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12TU4zvu38mF1fNSergCtMj2cK9eu57Rdfpt5MVk3tc4MZF6dNr, address:104.36.82.34:4558): protocol: rpc: dial tcp 104.36.82.34:4558: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582856475 (202h57m20.013936606s) [noise=true , address=104.36.82.35:4594 , piece_id=6NUSPNYGSSJOB2CNPIW6DI45MPZARLWDBXH3CWMP4LXMDB2CXWZQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:12D9nh4XFmsjjFcZEMjaL4uw4Whsa8nfMrAsYPu6NVpsxgb3jvf, address:104.36.82.35:4594): protocol: rpc: dial tcp 104.36.82.35:4594: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.582973238 (202h57m20.014053369s) [noise=true , address=104.36.82.19:4436 , piece_id=KVDJTKON4LFVJSAY7XSJ2OQ4ZH67JLMDE2O4EHUWSMZ3EECWSOXA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1WvyEw8XevCiQ8yRgzyMo7WTVzpEJ158ErKfAeV4dSTBkS1oRF, address:104.36.82.19:4436): protocol: rpc: dial tcp 104.36.82.19:4436: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583033393 (202h57m20.014113524s) [noise=true , address=104.36.82.58:4283 , piece_id=LJKEH2KDSOFPVFPB5UWGVLOHQVZQOY33MYSHFLP4ZWGPKZ4KC64A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1p9jkPtQnmbCXBmw6HSHDWpGaKWqfg87m6hHXy6WDu2giAFnbQ, address:104.36.82.58:4283): protocol: rpc: dial tcp 104.36.82.58:4283: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583069241 (202h57m20.014149362s) [noise=true , address=104.36.82.50:4047 , piece_id=AKDACFQEPBAFGYKTWBL5RELLZL3L4IJHJT73NS7T7QPWK2C2TDQA, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:126Z6tvp49dLLdj3T5snJc4tBVGzcX5UBaGafFN3PkhF92vxpfs, address:104.36.82.50:4047): protocol: rpc: dial tcp 104.36.82.50:4047: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583076716 (202h57m20.014156837s) [noise=true , address=104.36.82.39:4753 , piece_id=R7BHT5RYLSCDZLTDDRXTSJFSRPLM5PNBYC6Q2G33C5SYIUCNVCLQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1zzW2z6noGQT6xdSmwSQBNYwip6gZjnmqZg7qdb6BP3nHyCLFz, address:104.36.82.39:4753): protocol: rpc: dial tcp 104.36.82.39:4753: i/o timeout; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583107535 (202h57m20.014187666s) [noise=true , address=104.36.82.23:4474 , piece_id=RITPHITVW655UQETT6UKS7BZO2S3F4P2MJIO4REOUKYYP4E2CDVQ, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1Fpshye6yGAo3yf71eVpxrCdvoRjpkGX9WEX8KCWYbvuhy5i9j, address:104.36.82.23:4474): protocol: write tcp 188.166.225.170:36642->104.36.82.23:4474: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583165355 (202h57m20.014245487s) [noise=true , address=104.36.82.5:4855 , piece_id=3FCDYA6CRYOOU3MDFJPEC4YQNICFVCN2A47RIYPSR4ZX5TIXZX3A, seg_pos={0 0}, upload=211101]: Done uploading piece. err: ecclient: upload failed (node:1KDNuE7YY8smhDjotushQvY3y9NtLxigbJELHD4Edxy8Qi4BWh, address:104.36.82.5:4855): protocol: write tcp 188.166.225.170:47052->104.36.82.5:4855: use of closed network connection; conn closed: op; piecestore: piecestore close: conn closed: op Aug 17 04:23:25.583194000 (202h57m20.014274131s) [seg_pos={0 0}, upload=211101]: Done waiting for segment. successful: 10 optimal: 14 errs: <nil> Aug 17 04:23:25.810738626 (202h57m20.241818757s) [upload=211101]: Done uploading segments... Aug 17 04:23:25.810761991 (202h57m20.241842122s) [upload=211101]: Upload finished. err: failed to upload enough pieces (needed at least 14 but got 10) Duration: 6.030657363s core uplink us-select-1 us1-placement-20
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [.....................X.....X.....................XX.........] duration: 1.832811835s FAIL (2025-08-17 05:03:53 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.832841913s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [XXXX........................................................] duration: 6.414801333s FAIL (2025-08-17 05:03:50 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.414824967s edge gateway us-select-1
checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline) antiblip: 4 out of 60 is failed (threshold=3) [......................X.....X.....................XX........] duration: 1.884767288s FAIL (2025-08-17 05:03:23 UTC) orbiter warning: operation: DownloadObject ======ERROR====== uplink: object not found ("orbiter_us_0/1K") ======LOGS====== Aug 17 04:56:50.046028598 (203h30m44.477108729s) [upload=211693]: Starting upload Aug 17 04:56:50.046078113 (203h30m44.477158244s) [upload=211693]: Done starting upload Aug 17 04:56:50.046127948 (203h30m44.477208079s) [upload=211693]: Uploading segments... Aug 17 04:56:50.046165199 (203h30m44.477245331s) [upload=211693]: Got next segment. Inline: true Aug 17 04:56:50.046169808 (203h30m44.477249929s) [upload=211693]: Waiting for error group managing segments... Aug 17 04:56:50.046201128 (203h30m44.477281249s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectBegin Aug 17 04:56:50.046204875 (203h30m44.477285006s) [upload=211693]: Flush batch item: pb.BatchRequestItem_SegmentMakeInline Aug 17 04:56:50.046207780 (203h30m44.477287911s) [upload=211693]: Flush batch item: pb.BatchRequestItem_ObjectCommit Aug 17 04:56:50.545849463 (203h30m44.976929594s) [upload=211693]: Done uploading segments... Aug 17 04:56:50.545868880 (203h30m44.976949011s) [upload=211693]: Upload finished. err: <nil> Duration:* 1.884795331s core uplink us-select-1 us1-placement-20 metainfo-projects
checkers.Gateway{"https://s3.us-select-1.storjapi.io/", AuthURL: "https://auth.us-select-1.storjshare.io/", Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "gateway-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (gateway.us_select_1.remote) antiblip: 4 out of 60 is failed (threshold=3) [.XXXX.......................................................] duration: 6.120583406s FAIL (2025-08-17 05:03:14 UTC) orbiter warning: operation error S3: PutObject, https response error StatusCode: 0, RequestID: , HostID: , canceled, context deadline exceeded Duration: 6.120604186s edge gateway us-select-1

Orbiter Check History

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-us-select", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us_select_1.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "uplink-tests/from_do_prod_ops_orbiter_ap_0/4M", Size: 4.0 MiB} failure (antiblip window 3/60): (uplink.us1.placement_20.remote)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)

checkers.Uplink{Satellite: "12EayRS2V1kEsWESU9QMRseFhdxYxKicsiFmxrsLZHeLUtdps3S@us1.storj.io:7777", Bucket: "uptime-checks-placement-20", Key: "orbiter_us_0/1K", Size: 1.0 KiB} failure (antiblip window 3/60): (uplink.us1.metainfo_projects.inline)